In Part 1 I installed Regional Harbor and the Argo CD Service on VCF 9.1.1, and created an Argo CD instance with VCF Automation sign-in. Part 2 puts it to work.
The idea is simple to state. A DevOps user on VCF writes Kubernetes manifests, pushes them to Git, and gets a virtual machine, some pods, a whole Kubernetes cluster and an application running inside that cluster, without opening vCenter or NSX Manager and without holding a kubeconfig for the new cluster.
Four kinds of thing, one API, one Git workflow.
This post walks through how it fits together, the details worth getting right, and then a second application deployed and tuned entirely from the Argo CD GUI.
The application and where each tier runs
I used Yelb, a small restaurant-voting app that splits cleanly into tiers:
Tier
Runs as
Created from
Database (Postgres)
A VM Service virtual machine, Ubuntu 24.04, built by cloud-init
apps/ five pointers for Argo CD
10-yelb-db.yaml
20-yelb-appserver.yaml
30-yelb-vks-cluster.yaml
40-yelb-registration.yaml
50-yelb-ui-appset.yaml
infra/
db-vm/ VirtualMachine + cloud-init
appserver/ Deployments + LoadBalancer Service
vks-cluster/ Cluster
registration/ ManagedEntity
workloads/
yelb-ui/ Deployment + LoadBalancer Service
The only command run by hand is kubectl apply -f apps/. That creates four Argo CD Applications and one ApplicationSet. No infrastructure exists at that point; Argo CD pulls the repo and the VCF controllers build the rest.
Three of these kinds come from VCF: VirtualMachine, Cluster with the VKS ClusterClass, and ManagedEntity. That is the point: the platform’s own resources are Kubernetes objects, so they fit the same workflow as a Deployment. The VM class, image and storage policy names are the ones your platform team has assigned to the namespace.
The part worth showing live: a cluster that registers itself
Git never contains the new cluster’s address or credentials. Two small pieces connect it. A ManagedEntity points at the Cluster and asks for a label:
The Argo CD Service sees it, registers the cluster in Argo CD, and puts app: yelb on the cluster secret. An ordinary upstream ApplicationSet then deploys the UI to every cluster carrying that label:
The repository already has these built in. If you adapt it to your own environment, these are the details that matter.
Assign the storage policy to the namespace. A policy that exists on the Supervisor is not enough. The VM and the Cluster are admitted only with a storage class that is assigned to the namespace they are created in.
Give vSphere Pods an explicit CPU request. A vSphere Pod’s CPU request becomes a resource-pool reservation, and with no request set it takes the value of the limit. requests: { cpu: "0" } keeps small pods from reserving capacity they do not need.
After changing a manifest, start a fresh sync. An automated sync that is retrying stays on the commit it started with. Terminate the running operation and sync again to pick up the new commit.
Connect the tiers with hostAliases. Argo CD 3.x leaves EndpointSlices alone by default, so they are not a good bridge between runtimes. A hostAliases entry in the Deployment is plain, visible and managed like everything else.
Reach the database on the VM’s own address. The app server in a vSphere Pod talks to Postgres on the VM’s IP. The address lives in Git, so update it if you rebuild the VM.
Check the ManagedEntity once the cluster is ready. If it was created before the cluster finished building, touch it with an annotation and it reconciles within a minute.
Match the database’s authentication to the app’s client. Yelb’s database client predates SCRAM, the Postgres 16 default, so the cloud-init sets md5 for this app.
The last one shows the nicest property of the whole setup. To apply a new cloud-init, commit the change and run kubectl delete vm yelb-db. Argo CD notices the VM is missing and creates a new one from Git, with the new cloud-init. Rebuilding a server becomes a one-line operation.
The result
Everything Healthy and Synced.The UI in VKS, calling the app server in a vSphere Pod, reading restaurants from Postgres on the VM.Who talks to whom.
Day 2: more capacity is a commit
Next I wanted a bigger worker for a larger application. That was a few lines added to cluster.yaml and a push:
Argo CD synced the change and VKS added the node pool. The Cluster carries a Prune=false annotation, so Git can change it but removing the file would not delete it.
All three clusters in the namespace Available.
A twelve-service shop from the Argo CD GUI
Not everything needs a repository on day one. I deployed Google’s Online Boutique demo, twelve services, entirely from Argo CD’s New App form: no commit, no kubectl, no Helm CLI.
The cluster that registered itself is simply there to pick.Synced within seconds; Progressing while images pull.
NSX handed out a new load-balancer address, and the shop was up.
Served from yelb-vks through an NSX load balancer.
Tune it and change it from the same GUI
Two changes followed, both made under Details → Parameters → Edit in Argo CD.
The first was tuning. One of the twelve services, emailservice, needed more CPU than the chart’s default 200m limit to start inside its liveness window. I raised emailService.resources.limits.cpu to 1, and Argo CD replaced that one pod.
The second was a feature flag. Setting frontend.cymbalBranding to false rolled only the frontend and changed the shop’s branding.
Editing a Helm parameter in place.Two edits, two pods replaced. The other ten never restarted.After the branding change. The cart kept its item, because the cart lives in Redis.An order, end to end.
Git or GUI? Yelb lives entirely in Git. The Online Boutique application and its overrides live in Argo CD. The GUI is fast for an experiment; for anything that should last, move the overrides into a values file in a repository.
What this adds up to
One repository built a VM, vSphere Pods and a VKS cluster. The cluster registered itself, and the application arrived without anyone attaching anything. A second application went in from a browser and was tuned from the same place.
For a platform team, that is a private, declarative slice of the private cloud for each team. Requests become pull requests, drift is corrected from Git, and every change has an author and a diff. For the teams using it, virtual machines, pods and clusters all arrive through the one workflow they already know.
The repository is public. Clone it, change the names to your own namespace, image and storage policy, and try it.
If you administer vSphere, you already know how to build a VM. This post shows you how to build the same VM in VMware Cloud Foundation Automation using nothing but a command line — no UI clicks, no cloud template, no prior Kubernetes experience assumed. It takes about less then 10 minutes, and by the end you’ll have a running web server with a load-balanced IP that answers from your browser.
I’m deliberately keeping this simple. There is a longer, deeper reference post for the full API surface, but this one is the on-ramp: the shortest honest path from “I’ve never used the VCF Automation API” to “I just built something with it.”
What we’re going to do
Install the CLI and log in to your organization
Create a project (the tenancy boundary for a team)
Create a Supervisor Namespace (where workloads actually live)
Create a VM with cloud-init that installs nginx
Publish it behind a load balancer and browse to it
Build a Kubernetes cluster in the same namespace, and run a workload on it
Why bother, when the UI works fine?
Fair question, and I’d give three honest answers.
Repeatability. A VM you clicked into existence is a VM nobody can rebuild exactly. A VM defined in a YAML file is one you can commit, review, and recreate identically a year later.
It’s where the platform is going. In VCF 9, the consumption layer is a Kubernetes-style API. The UI is a client of that API, not the other way round. Learning the objects underneath means the UI stops being magic.
It’s the foundation for automation. Everything you’d want later — GitOps, pipelines, self-service catalogs — is built on these same objects. This is the first rung.
None of that means you should stop using the UI. I use both. But knowing what’s underneath makes you considerably harder to stump.
Before you start
What
Where to get it
VCF CLI (the vcf command)
Broadcom Support portal — it replaces the older CCI kubectl plugin
kubectl
Any recent version; the CLI generates a kubeconfig for it
An API token
Log into your VCF Automation org portal → your user menu → API Tokens → Generate. Copy it — it’s shown once.
Org admin rights
Needed to create projects. If you only have consumer rights, skip to Step 4 and use an existing namespace.
Everything below is run from PowerShell on Windows. It works identically on Linux and macOS; only the quoting changes.
Step 1 — Log in
First, store your token in a variable. Note the quotes — without them PowerShell tries to execute the token as a command, which produces a confusing error.
$vcfa_token = "your-api-token-here"
Now create a context. This is the CLI’s word for “a saved connection to an environment.”
Substitute your own endpoint and tenant name. The backtick is PowerShell’s line-continuation character — or just put it all on one line. Use --insecure-skip-tls-verify only in a lab; in production supply your CA certificate with --ca-certificate instead.
A successful login. Note that the CLI lists every context you now have access to — one for the organization, plus one for each Supervisor Namespace you can reach.
That output is worth reading carefully, because it explains the whole model. You get an organization context (org2-ctgw) and a namespace context for each namespace you can use (org2-ctgw:org2-ns1-4hm32:default-project).
The one concept to take away
There are two levels. The organization context is where you manage tenancy — projects, namespaces, who gets what. The namespace context is where you build things — VMs, clusters, networks. Almost every “why can’t I see that?” moment is being in the wrong one. If a command returns “the server doesn’t have a resource type”, you’re at the org level asking a workload question.
Step 2 — Create a project
A project is the boundary you’d hand to a team. Save this as project.yml:
apiVersion: project.cci.vmware.com/v1alpha2
kind: Project
metadata:
name: team-phoenix
spec:
description: Phoenix dev team
kubectl --context org2-ctgw create -f project.yml
# project.project.cci.vmware.com/team-phoenix created
Use create, not apply
This will save you a puzzling ten minutes. kubectl apply stores a tracking annotation on the object, and this API rejects it outright with “Annotation updates are not supported.” Everywhere in this post, use kubectl create.
Step 3 — Create a namespace
The project is just a boundary. A Supervisor Namespace is the thing with actual capacity — CPU, memory, storage, a network — where your VMs will live. Think of it as the resource pool plus the network plus the quota, expressed as one object.
You need four pieces of information, all of which you can read off an existing namespace or get from whoever set up your environment:
Two things that catch people here
The namespace name is generated with a random suffix (phoenix-ns-kj9g6), so read it from the previous command rather than guessing. And the context name is the full three-part string <org>:<namespace>:<project> — using just the namespace name gives you “no context exists.”
You can confirm the same thing in the UI, which is a reassuring way to check your work while you’re learning:
The namespace created from the command line, shown in the VCF Automation UI — Active, in the team-phoenix project, with its CPU, memory and storage allocation. The API and the UI are two views of the same object.
Why the long classConfigOverrides block?
When you create a namespace through the portal, it fills these values in for you from the class. Creating one through the API, you have to state them yourself — and the zone entry needs all four CPU and memory fields, not just a name. If you leave them out, the API tells you exactly what’s missing, one field at a time. That’s actually a decent way to learn the object: submit a minimal version and let the error messages guide you.
Step 4 — Create a VM
Now the part that will feel familiar. We need three things: a size, an image, and a storage policy. Ask the namespace what it has:
kubectl get virtualmachineclasses # the sizes available to you
kubectl get virtualmachineimages # images from your content library
kubectl get clustervirtualmachineimages | Select-String "ubuntu"
If virtualmachineimages comes back empty, you’re not stuck.
That list is empty when no content library is attached to your namespace class. You can still deploy from a cluster-wide image by referencing its vmi- identifier directly, which is what I do below. Getting a content library attached is the better long-term answer — it gives you friendly names instead of hashes — but it isn’t a blocker today.
Here’s the VM, together with a cloud-init secret that creates a login and installs nginx. Save both in one file:
Two objects, one command: the cloud-init secret and the VM itself.
Watch it come up. The VM will report an IP address once VMware Tools starts reporting:
kubectl get vm web01
kubectl get vm web01 -o jsonpath='{.status.network.primaryIP4}'
# 172.30.0.66
Notice what you didn’t have to specify: no port group, no IP address, no datastore. The VM picked up an address from the namespace’s default network automatically and landed on storage matching the policy you named. That’s the platform doing the placement work, exactly as it does for a Kubernetes pod.
Cloud-init only runs on first boot
You cannot add a bootstrap section to a VM that already exists. If you forget it, the VM boots fine but has no user account and no way in — you delete and recreate. Decide your bootstrap before you provision.
Step 5 — Publish it
A VirtualMachineService puts a load-balanced IP in front of one or more VMs, selected by label. Our VM carries app: web, so the service will find it.
The load balancer service, created. NSX allocates the external address within seconds.
Reading the external IPkubectl get vmservice prints only name, type and age — the address isn’t in its default columns. Use kubectl get svc instead, or ask for the field directly with -o jsonpath='{.status.loadBalancer.ingress[0].ip}'.
Step 6 — Confirm it works
kubectl get svc
# web-lb LoadBalancer 172.29.255.24 40.60.0.4 80/TCP 2m31s
kubectl get endpoints
# web-lb 172.30.0.66:80 2m34s
curl.exe http://40.60.0.4
The whole chain proven: external IP 40.60.0.4, backend 172.30.0.66:80, and nginx answering.
That’s the full path working. The load balancer address is allocated from the NSX VPC, the endpoint is your VM’s own IP, and nginx — installed by cloud-init at first boot — is serving. Nothing in that sequence involved the UI.
Step 7 — Build a Kubernetes cluster
Here’s the part that surprises people: a Kubernetes cluster is just another object in the same namespace you’ve been working in. Same context, same kubectl create, same mental model. You describe the cluster you want and the platform builds the nodes.
You need to know two things first — and both are places where a sensible-looking manifest fails.
Where the cluster template lives. Clusters are built from a ClusterClass, which is a template your platform team maintains. Crucially, it does not live in your namespace:
kubectl get clusterclasses
# NAME VARIABLES READY AGE
# builtin-generic-v3.1.0 True 164m
# builtin-generic-v3.2.0 True 164m
# builtin-generic-v3.3.0 True 164m
That list is not the whole story. The class you actually want may live in a shared system namespace and won’t appear here at all — you reference it explicitly by name and namespace.
Which Kubernetes version you can use. Ask the platform rather than guessing:
kubectl get kubernetesreleases
You’ll get a long list. Only the rows showing bothREADY: True and COMPATIBLE: True are usable. Pick one of those.
Now the cluster itself — one control plane node, one worker, deliberately small:
kubectl create -f cluster.yml
kubectl get cluster phoenix-k8s -w
Provisioning takes five to six minutes for this size. PHASE: Provisioned appears early and only means the objects were created — the column to watch is AVAILABLE, which flips to True once the control plane is genuinely up. kubectl get machines gives you the per-node view while you wait.
Three things in that manifest are not obvious1.classRef needs a namespace. The ClusterClass lives in a shared system namespace, and without it you’ll get a “not found” even though the class plainly exists. 2. The version string is the long form — v1.35.5---vmware.1-vkr.1, with three dashes, not the +vmware.1 form shown elsewhere. 3. The resolve-os-image annotation is often required. If your environment has the same node image registered in more than one content library — common in labs that have been upgraded a few times — the resolver refuses to guess and fails with “Multiple OSImages resolved.” Adding content-library= narrows it to one. You’ll find the library ID in your Content Libraries list.
The shortcut worth knowing
Rather than assembling this by hand, start a cluster creation in the VCF Automation UI and look at the YAML it generates. It will show you the right ClusterClass namespace, the right version format, and any annotations your environment needs — all correct for your specific build. Copy that, then edit it. This is far quicker than reverse-engineering from error messages, and it’s how I arrived at the manifest above.
Step 8 — Get onto the cluster and run something
Once AVAILABLE is True, download the cluster’s kubeconfig from the VCF Automation UI (the cluster’s detail page offers it) and point kubectl at it:
kubectl --kubeconfig .\phoenix-k8s.yaml get nodes
# NAME STATUS ROLES AGE VERSION
# phoenix-k8s-... Ready control-plane 12m v1.35.5+vmware.1
# phoenix-k8s-...-np-1-... Ready <none> 9m v1.35.5+vmware.1
You’re now a cluster administrator on an ordinary, conformant Kubernetes cluster. Everything you know about Kubernetes applies from here.
With one important exception, which is worth meeting deliberately rather than by accident.
Your first deployment will probably be rejected — and that’s correct
Try the obvious thing:
kubectl --kubeconfig .\phoenix-k8s.yaml create deployment web --image=nginx --replicas=2
The deployment is created, but no pods ever appear. Looking at the ReplicaSet tells you why:
pods "web-..." is forbidden: violates PodSecurity "restricted:latest":
allowPrivilegeEscalation != false
unrestricted capabilities (must set capabilities.drop=["ALL"])
runAsNonRoot != true
seccompProfile (must set seccompProfile.type to "RuntimeDefault")
This is Pod Security Admission enforcing the restricted profile, and it’s switched on by default. A stock container image that runs as root is refused at creation — not warned about, refused. For a vSphere admin used to “deploy first, harden later,” this is a genuine change of habit, and a welcome one: the cluster will not run a privileged workload just because you forgot.
The fix is to declare what the workload actually needs. Each field below answers one of the four complaints, and the image is swapped for a non-root variant that listens on 8080 rather than binding port 80 as root:
kubectl --kubeconfig .\phoenix-k8s.yaml apply -f web.yml
kubectl --kubeconfig .\phoenix-k8s.yaml expose deployment web --port=80 --target-port=8080 --type=LoadBalancer
kubectl --kubeconfig .\phoenix-k8s.yaml get svc web
The external IP is allocated by the same NSX load balancer that served your VM in Step 5 — usually within seconds. Confirm there are backends behind it:
Note that kubectl apply works here.
The “use create, not apply” rule applies to the VCF Automation API only. Inside a VKS cluster you’re talking to a normal Kubernetes API server, and all your usual habits — apply, Helm, Kustomize — work exactly as expected.
The five things that will trip you up
These cost me time so they don’t have to cost you any.
Use kubectl create, never apply. The API rejects the annotation that apply depends on.
Context names are three parts.<org>:<namespace>:<project>, not just the namespace.
Know which level you’re at. “The server doesn’t have a resource type” almost always means you’re at the org context asking a workload question, or vice versa.
Tokens expire mid-session. A sudden Unauthorized, or a strange complaint about downloading the OpenAPI schema, usually just means your session lapsed. vcf context use <org> re-authenticates.
Check the API version against your build. Published examples lag. On the environment used here, namespaces are served at v1alpha3 and VMs at v1alpha5. The most reliable way to learn the current shape of any object is to dump a working one: kubectl get vm <name> -o yaml. Copy the structure, change the values.
That last one is worth internalising as a general habit. Rather than hunting for documentation that matches your exact release, clone something that already works in your own environment. It is always correct, by definition.
Where to go next
You now have the pattern for everything else on this platform. The same two-level model, the same create workflow, and the same “clone a working object” trick apply to Kubernetes clusters, persistent volumes, NSX security policies, and micro-segmentation rules — all of which are just more objects in the same API.
A few natural next steps, in the order I’d take them:
Delete and rebuild what you just made. Change the VM size in the YAML, recreate, and watch it come back identically. That’s the payoff of declarative infrastructure, and it’s more convincing when you do it than when you read about it.
Scale the cluster. Change replicas on the worker pool, re-apply, and watch nodes roll in. Then bump version to a newer compatible release and watch a rolling upgrade — that’s a one-line change doing something that used to be a project.
Write a security policy as YAML and watch NSX distributed firewall rules appear.
Put it all in Git. Once your infrastructure is files, a GitOps engine can reconcile it. That’s the subject of my next post.
If you’re a vSphere admin who has been eyeing all this with mild suspicion, I’d encourage you to run through the steps above once in a lab. The concepts are less foreign than the vocabulary suggests — a namespace is a resource pool with a network and a quota, a VM is still a VM, and a load balancer service is the thing you’d have raised a ticket for. The difference is that now it’s a file you own.
Questions or something that didn’t work in your environment? Leave a comment — I read all of them, and environment-specific quirks are exactly what makes these posts better.