Skip to content

Application layer

Pulumi Go stack that deploys the Shaide application into Kubernetes namespace app-shaide. This stack owns the application layer (Shaide server, control panel UI, RustFS object storage, and Qdrant vector database) and wires everything together with shared configuration and secrets managed through Pulumi.

Repository Layout

app_shaide/ itself only contains main.go (a thin wrapper that calls shaide.DeployAppShaide(ctx)) and deployments/. The actual implementation lives in the shared pkg module, at pkg/iac/shaide/:

pkg/iac/shaide/
├── shaide.go                              # DeployAppShaide — orchestration entry point
└── internal/
    ├── config/config.go                   # Config loading; typed view of Pulumi stack config
    ├── runtime/context.go                  # DeploymentContext — shared labels + dependency options
    ├── platform/
    │   ├── k8s-serviceaccount.go           # shaide-server ServiceAccount (+ workload-identity annotations)
    │   ├── k8s-rbac.go                     # ClusterRole/ClusterRoleBinding — cluster-wide pod/service/namespace read
    │   └── configmap.go                    # shaide-config ConfigMap + shaide-secrets Secret
    ├── components/
    │   ├── shaide/deploy.go                # shaide-server StatefulSet + Service (+ HTTPRoute)
    │   ├── controlpanel/deploy.go          # control-panel Deployment + Service
    │   ├── webapp/deploy.go                # webapp Deployment + Service
    │   ├── rustfs/deploy.go                # rustfs StatefulSet + Service
    │   └── qdrant/deploy.go                # qdrant StatefulSet + Service
    └── cloudprovider/                      # Provider interface + per-cloud implementations
        ├── provider.go, gcp.go, aws.go, azure.go, on-prem.go, generic.go

What Each Component Does

main.go

Compiles the stack binary and delegates to shaide.DeployAppShaide(ctx).

pkg/iac/shaide/shaide.go

The stack orchestrator and dependency coordinator: - Loads Pulumi config via appconfig.Load(ctx). - Creates the Kubernetes provider used by all resources in this stack. - Creates the namespace first, so all subsequent resources are scoped correctly. - Creates shared prerequisites: shaide-config ConfigMap, shaide-secrets Secret, ServiceAccount, and cluster-wide RBAC. Images are pulled anonymously, so no pull secret is created. - Selects a cloudprovider.Provider (see below) and calls ProvisionStorage before any StatefulSet is created. - Calls per-component Deploy functions in a stable order: shaide, controlpanel, webapp, rustfs, qdrant. - Threads shared dependencies through a runtime.DeploymentContext so resources only create after their inputs are ready.

pkg/iac/shaide/internal/platform/

  • k8s-serviceaccount.go: Creates the ServiceAccount used by shaide-server (shaideServiceAccountName). Attaches whatever annotations are set in serviceAccountAnnotations — a generic map, so the same code handles GKE Workload Identity, AKS Workload Identity, EKS IRSA, or nothing at all (on-prem/generic).
  • k8s-rbac.go: Creates a cluster-scoped ClusterRole/ClusterRoleBinding granting shaide-server's ServiceAccount get/list/watch on pods, services, and namespaces cluster-wide — this is what lets Shaide observe pod state and discover installed-model topology across every namespace it needs to (its own, app_serving's per-model namespaces, app_mcp's namespace, etc.), not just its own namespace. See Installed Model Discovery below.
  • configmap.go: Creates the shared shaide-config ConfigMap (non-secret runtime settings, service discovery) and shaide-secrets Secret (adminAuthKey, s3Password, jwtSecret, sessionSecret).

pkg/iac/shaide/internal/components/

  • shaide/deploy.go: Deploys shaide-server as a StatefulSet with a PVC mounted at /root/.config for SQLite persistence, and exposes it through a ClusterIP Service. When infraStackRef or gatewayHostname is set, an HTTPRoute attaches the Service to the shared Gateway. shaide-server is never a LoadBalancer of its own. Delegates cloud-specific post-deploy resources to the active cloudprovider.Provider.
  • controlpanel/deploy.go: Deploys the control panel as a Deployment (single replica, no persistence) with a ClusterIP Service on port 3000. Receives SESSION_SECRET from the shared shaide-secrets Secret, and KNOWLEDGE_CENTER_ENABLED ("true"/"false", from knowledgeCenterEnabled; default: false) indicating whether the Knowledge Center feature is present. It has no direct knowledge of installed models — it consumes shaide-server's own model API. See Installed Model Discovery below.
  • webapp/deploy.go: Deploys the end-user facing web application as a Deployment (single replica, no persistence) with a ClusterIP Service on port 8787. Same shape as the control panel — internal-only, points at shaide-server via SHAIDE_SERVER_FQDN/SHAIDE_SERVER_PORT env vars.
  • rustfs/deploy.go: Deploys rustfs as a StatefulSet with a PVC for /data, an emptyDir for /logs, and a ClusterIP Service on port 9000 (plus 9001 when rustfsConsoleEnabled is set). The container runs as UID/GID 10001 and is configured via values from the shared ConfigMap and Secret.
  • qdrant/deploy.go: Deploys qdrant as a StatefulSet with a PVC for /qdrant/storage and a ClusterIP Service exposing REST 6333 and gRPC 6334.

pkg/iac/shaide/internal/cloudprovider/

A Provider interface (provider.go) with one implementation per target, selected by the informational cloudProvider config value:

cloudProvider ProvisionStorage PostDeployService
gcp no-op (GKE pd.csi.storage.gke.io dynamic provisioner) Creates a GKE HealthCheckPolicy targeting /v1/health
azure no-op (AKS disk.csi.azure.com dynamic provisioner) no-op (Workload Identity pod label is applied directly in shaide/deploy.go)
aws no-op (EBS CSI dynamic provisioner) no-op
on-prem Creates one static, pre-bound hostPath PersistentVolume per stateful component (shaide-server, rustfs, qdrant), pinned to pvNodeHostname — only when storageClassName is hostpath no-op (MetalLB gives the shared Gateway its address; shaide-server stays ClusterIP)
anything else no-op no-op

Any unrecognized cloudProvider value falls back to the generic no-op provider — useful for local/dev clusters that already have a working default StorageClass and don't need a LoadBalancer at all.

Component Ports

Component Service Name Ports Scope
Shaide server shaide-server 80 -> 8080 External only through the shared Gateway (HTTPRoute)
Control panel control-panel 3000 Internal only
Web app webapp 8787 Internal only
RustFS rustfs 9000 (9001 when app_shaide:rustfsConsoleEnabled=true) Internal only
Qdrant qdrant 6333, 6334 Internal only

emptyDir Permissions Note (RustFS)

emptyDir does not support direct permission or ownership settings in pod spec. The fix is an initContainer (defined in pkg/iac/shaide/internal/components/rustfs/deploy.go) that prepares the filesystem before the main container starts: - Runs as root. - Sets /data and /logs to mode 0755. - Sets ownership to 10001:10001. - Starts the main rustfs container after permissions are correct.

This matches RustFS expectations (0o755) and avoids runtime permission errors.

Installed Model Discovery

This stack does not hand the Control Panel a model list. Responsibilities are kept with their actual sources:

  • Kubernetes / app_serving: topology and routing. Each model's routable Service (llmd-gateway-<slug> for generative, ms-<slug>-embeddings for embedders) is labeled with axem.dev/model-slug, axem.dev/model-category, and app.kubernetes.io/part-of=app-serving — see Model.MetaLabels() in pkg/iac/serving/internal/config/naming.go.
  • vLLM: model-owned metadata (name, context size, ...), served by each model's own /server-info endpoint.
  • shaide-server: combines the two at runtime — discovers model Services cluster-wide via the labels above (k8s-rbac.go grants the services/ namespaces read access this needs) and queries each one's /server-info — and exposes a stable model API.
  • Control Panel: consumes that API only; it has no model-related config or mount of its own.

This stack's only job for installed models, then, is granting shaide-server the RBAC to discover them (k8s-rbac.go) — it carries no model list.

Configuration Source of Truth

All parameters are defined in the active stack's deployments/Pulumi.<stack>.yaml. Each stack config is the authoritative list of settings for that deployment target, including: - Namespace and platform behavior (app_shaide:namespace, app_shaide:cloudProvider, app_shaide:infraStackRef, app_shaide:nodeSelector, app_shaide:shaideServiceAccountName). - Container images (shaideServerImage, controlPanelImage, webappImage, rustfsImage, qdrantImage). - Runtime config (shaideServerS3Fqdn, shaideServerS3Port, databaseUrl, vectorDBUrl). - Trial deployment flag (trial, defaults to FALSE; injected into shaide-server as the TRIAL env var — only the trial stack sets it to TRUE). - RustFS console exposure (rustfsConsoleEnabled). - MCP integration (mcpNamespace, optional — see MCP Integration). - Secrets (adminAuthKey, s3Password, jwtSecret, sessionSecret).

The nodeSelector value is rendered as a soft (preferred, not required) nodeAffinity for a nodegroup label on the target node pool (e.g. shaide-nodepool) — components prefer a matching node but can still schedule elsewhere if none is available. Each component also gets a soft podAntiAffinity spreading its own replicas across nodes.

Prereqs

  • Kubernetes context points to the target cluster.
  • Pulumi stack is selected for this project.
  • Required secrets are set (adminAuthKey, s3Password, jwtSecret, sessionSecret).

Workload Identity

app_shaide:serviceAccountAnnotations is a generic annotation map applied to the shaide-server ServiceAccount — the same mechanism works for any cloud's workload identity binding:

  • GKE Workload Identity: iam.gke.io/gcp-service-account: <gsa-email>
  • AKS Workload Identity: azure.workload.identity/client-id: <managed-identity-client-id> (Azure additionally requires the azure.workload.identity/use: "true" pod label, which is applied automatically whenever cloudProvider: azure.)
  • EKS IRSA: eks.amazonaws.com/role-arn: <role-arn>
  • On-prem/generic: omit serviceAccountAnnotations entirely.

app_shaide:shaideServiceAccountName (defaults to shaide-server) names the ServiceAccount these annotations are applied to.

Required IAM on the mapped cloud identity (for Vertex AI access): roles/aiplatform.user or the equivalent role on the target cloud.

MCP Integration (optional)

app_shaide:mcpNamespace points shaide-server at the namespace where the app_mcp stack is deployed (e.g. mcp-gateway). It is optional: - If set, MCP_NAMESPACE is injected into shaide-config and shaide-server can discover/watch MCP pods in that namespace. - If left unset, MCP_NAMESPACE is omitted from shaide-config entirely and shaide-server runs without MCP support — no app_mcp deployment is required.

Security Notes

  • Sensitive values live in the shaide-secrets Kubernetes Secret created by Pulumi.
  • Every image registry (the in-cluster Harbor mirror and the upstream registries) allows anonymous pulls, so the stack holds no registry credentials.
  • Avoid committing plaintext secrets into stack config; use pulumi config set --secret.

Resource Ownership

This stack owns only the app-shaide application layer resources. Cluster-level routing, Gateways, and cloud infra are expected to be managed by the infra stacks referenced via infraStackRef.

Data Persistence

Persistent storage is provided by PVCs for Shaide SQLite (/root/.config), RustFS (/data), and Qdrant (/qdrant/storage). Deleting PVCs will permanently remove stored data.

Gateway Mode

The shaide-server Service is always ClusterIP. External traffic reaches it only through the shared Gateway, whose load balancer the platform provides: a cloud load balancer, or MetalLB on-prem (see Gateway and routing and On-prem RKE2).

When either infraStackRef (a StackReference to an infra stack that exports gatewayHostname) or gatewayHostname (set directly) is non-empty, an HTTPRoute attaches shaide-server to the shared Gateway. The two are mutually exclusive in practice: set one or the other, not both. When neither is set, no HTTPRoute is created and shaide-server is reachable only from inside the cluster.

Local access for development

To reach shaide-server without a Gateway, for example on a development cluster without a load balancer, forward a local port to its Service:

kubectl -n app-shaide port-forward svc/shaide-server 8080:80

shaide-server is then served at http://localhost:8080. The forward lasts as long as the command runs. This is for development only; the installer does not set it up.

Config Changes

Update the active stack config (deployments/Pulumi.<stack>.yaml), then apply changes with pulumi up to reconcile the cluster.

Deploy

cd app_shaide
pulumi up

Quick Checks

# Namespace and workloads
kubectl get ns app-shaide
kubectl get pods -n app-shaide -o wide

# Services and endpoints
kubectl get svc -n app-shaide -o wide
kubectl get endpoints -n app-shaide

# ServiceAccount + WI mapping (GKE)
kubectl get pod -n app-shaide shaide-server-0 -o jsonpath='{.spec.serviceAccountName}{"\n"}'
kubectl get sa -n app-shaide shaide-server -o yaml | rg "iam.gke.io/gcp-service-account"
kubectl exec -n app-shaide shaide-server-0 -- sh -lc \
  'curl -s -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/email; echo'

# Shaide health (inside cluster)
# Replace service names if you customized them in Pulumi config.
kubectl run -n app-shaide shaide-health --rm -i --restart=Never --image=curlimages/curl:8.5.0 -- \
  curl -sS http://shaide-server/v1/health

# RustFS permissions and logs
kubectl exec -n app-shaide rustfs-0 -- ls -ld /data /logs
kubectl logs -n app-shaide rustfs-0

Troubleshooting

  • ImagePullBackOff for shaide-server:
  • Confirm the image reference in shaideServerImage exists in the registry it points at.

  • For a Harbor mirror, confirm the project is public (the Harbor stack creates them public).

  • RustFS fails with permission errors:

  • Ensure the fix-permissions initContainer ran and set /data and /logs to 0755 with 10001:10001.

  • Check initContainer logs: kubectl logs -n app-shaide rustfs-0 -c fix-permissions.

  • Shaide server not reachable:

  • Verify shaide-server Service type and endpoints: kubectl get svc,endpoints -n app-shaide.

  • If on GCP with infraStackRef, confirm HTTPRoute exists: kubectl get httproute -n app-shaide.

  • Vertex returns PERMISSION_DENIED:

  • Verify shaide-server uses the expected KSA and that KSA is annotated with iam.gke.io/gcp-service-account.

  • Ensure the mapped GSA has roles/aiplatform.user.

  • Qdrant not responding:

  • Check pod status and PVC binding: kubectl get pods,pvc -n app-shaide.
  • Confirm ports are open in the Service: kubectl describe svc -n app-shaide qdrant.