The page contains frequently asked questions about configuring and using the Deckhouse Platform.

Common

How can I apply an update for a specific module immediately?

To apply an update for a specific module immediately, set the modules.deckhouse.io/apply-now: "true" annotation on the corresponding ModuleRelease resource.

This annotation applies the release immediately without waiting for the update window. The requirements from spec.requirements still apply. If they are not met, the release will not be applied.

Example of setting the annotation for the console module:

d8 k annotate mr console-v1.43.3 modules.deckhouse.io/apply-now="true"

This can also be done using the d8 CLI for convenience (module names and versions are autocompleted):

d8 system module apply-now console v1.43.3

Example resource with the annotation set:

apiVersion: deckhouse.io/v1alpha1
kind: ModuleRelease
metadata:
  name: console-v1.43.3
  annotations:
    modules.deckhouse.io/apply-now: "true"
...

How can I apply an update DP immediately, bypassing update windows, canary releases, and manual update mode?

To apply a Deckhouse Platform (DP) update immediately, add the annotation release.deckhouse.io/apply-now: "true" to the corresponding DeckhouseRelease resource.

This will bypass update windows, canary release settings, and the manual cluster update mode. The update will be applied immediately after the annotation is set.

Example command to set the annotation and skip update windows for version v1.56.2:

d8 k annotate deckhousereleases v1.56.2 release.deckhouse.io/apply-now="true"

Example of a resource with the annotation set:

apiVersion: deckhouse.io/v1alpha1
kind: DeckhouseRelease
metadata:
  annotations:
    release.deckhouse.io/apply-now: "true"
...

What to do if, with VPN enabled, the container with the installer cannot access the network?

If a VPN is installed on the computer from which the cluster bootstrap is performed, there may be a problem with the container with the installer (or graphical installer) accessing the network. Because of this, for example, the graphical installer may not be displayed in the browser.

The problem can be solved in one of the following ways:

  • Disable VPN on your computer and restart the container with the installer.

  • If you cannot disable VPN (for example, if the cluster bootstrap is running on a VPN network), use the --network host parameter when starting the container with the installer (for Docker Desktop on Mac OS, the parameter is available starting with version 4.34.0). This will allow the container to access the network.

    Example of launching a container with an installer with VPN enabled, using the --network host parameter:

    docker run --network host --pull=always -it -v "$PWD/config.yml:/config.yml" -v "$HOME/.ssh/:/tmp/.ssh/" -v "$PWD/dhctl-tmp:/tmp/dhctl" registry.deckhouse.ru/deckhouse/ce/install:early-access bash

    Example of running the graphical installer with VPN enabled, using the --network host parameter:

    docker run --network host --rm --pull always -v $HOME/.d8installer:$HOME/.d8installer -v /var/run/docker.sock:/var/run/docker.sock -p 127.0.0.1:8080:8080 registry.deckhouse.ru/deckhouse/installer:latest -r $HOME/.d8installer

How do I change the release channel for a module?

A module can be built-in to DP or connected from a module source (defined using ModuleSource). Built-in modules have a common release cycle with DP and are updated together with DP. The release channel of a built-in module always matches the DP release channel. A module connected from a source has its own release cycle, which is independent of the DP release cycle. The release channel of a module connected from a source can be changed.

Below is the process of changing the release channel for a module connected from a source.

By default, the release channel for modules is inherited from the DP release channel (specified in the releaseChannel parameter of the deckhouse ModuleConfig). For more information about release channels, see the Release Channels section.

For modules from a source, the release channel is specified using ModuleUpdatePolicy, which is then linked to the module via the updatePolicy parameter in ModuleConfig.

To change the release channel for a module from a source, follow these steps:

  1. Define the module update policy.

    Create a ModuleUpdatePolicy where you specify the release channel in the releaseChannel parameter.

    Example ModuleUpdatePolicy:

    apiVersion: deckhouse.io/v1alpha2
kind: ModuleUpdatePolicy
metadata:
  name: my-module-policy
spec:
  releaseChannel: Alpha
  update:
    mode: Auto

    If necessary, change the update mode and configure update windows.

    Ensure that the policy has been created:

    d8 k get mup my-module-policy

    Output example:

    NAME               RELEASE CHANNEL   UPDATE MODE
my-module-policy   Alpha             AutoPatch

  2. Link the update policy to the module.

    Specify the name of the created update policy in the updatePolicy parameter of the corresponding module’s ModuleConfig.

    To edit the ModuleConfig, use the command (specify the module name):

    d8 k edit mc my-module

    Example ModuleConfig:

    apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: my-module
spec:
  enabled: true
  # ModuleUpdatePolicy name.
  updatePolicy: my-module-policy

When you change the module’s release channel, its version will change according to the configured update mode.

To view the current release channel of the module and other information about the module’s state in the cluster, use the corresponding Module object.

Example command to get information about the module:

d8 k get module my-module -o yaml

The update policy used will be specified in the properties.updatePolicy field, and the current release channel in the properties.releaseChannel field. Example output:

apiVersion: deckhouse.io/v1alpha1
kind: Module
metadata:
  name: my-module
  # ...
properties:
  # ...
  releaseChannel: Alpha # Module release channel.
  updatePolicy: my-module-policy # Module update policy.
  version: v1.16.10  # Module version.
  # ...

How do I configure an external module to update to patch versions only?

By default, if a module has no dedicated update policy, the update mode and windows are inherited from the DP settings.

If DP is set to the AutoPatch mode, the external module will also automatically receive only patch versions within the current minor version. Moving to a new minor version will require manual approval. In this case, no additional configuration is needed.

For more information about DP update modes, see Configuring updates.

If you need to manage the module update mode independently of DP, create a ModuleUpdatePolicy with update.mode: AutoPatch and link it to the module via the updatePolicy parameter in ModuleConfig:

  1. Create an update policy.

    Example ModuleUpdatePolicy:

    apiVersion: deckhouse.io/v1alpha2
kind: ModuleUpdatePolicy
metadata:
  name: my-update-policy
spec:
  releaseChannel: Stable
  update:
    mode: AutoPatch

    Ensure that the policy has been created:

    d8 k get mup my-update-policy

  2. Link the policy to the module.

    Specify the policy name in the updatePolicy parameter of the module’s ModuleConfig:

    d8 k edit mc module

    Example ModuleConfig:

    apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: module
spec:
  enabled: true
  updatePolicy: my-update-policy

In the AutoPatch mode, module patch versions (for example, from v1.16.1 to v1.16.2) are applied automatically, taking update windows into account if they are configured. To move to a new minor version (for example, from v1.16.* to v1.17.*), approve the corresponding ModuleRelease:

d8 k annotate mr module-v1.17.0 modules.deckhouse.io/approved="true"

Or using the d8 CLI:

d8 system module approve module v1.17.0

What should I do if the module image did not download and the module did not reinstall?

In some cases, there may be a problem with automatically downloading the image and reinstalling the module. These cases include:

  • Damage to the file system or other problems that have rendered the module image invalid.
  • Switching to a different registry.
  • Switching from one DP edition to another.

In this case, the module may be in the Ready state. The error occurs in the module’s pods. To find the problematic pod, use the command:

d8 k -n d8-<module-name> get pods

A problematic pod will have a status other than Running.

To view information about a pod, use the command:

d8 k -n d8-<module-name> describe pod <pod-name>

Example of an error message in the pod when there is a problem with downloading the image and reinstalling the module:

Failed to pull image "registry.deckhouse.ru/deckhouse/ce/modules/console@sha256:a12b4f8de1d997005155d0ba0a7c968a015dd8d18bb5d54645ddb040ddab1ef4": rpc error: code = NotFound desc = failed to pull and unpack image "registry.deckhouse.ru/deckhouse/ce/modules/console@sha256:a12b4f8de1d997005155d0ba0a7c968a015dd8d18bb5d54645ddb040ddab1ef4": failed to resolve reference ...

To download the image and reinstall the module that caused the problem:

  1. Get a list of module releases:

    d8 k get mr -l module=my-module

    Output example:

    NAME               PHASE        UPDATE POLICY   TRANSITIONTIME   MESSAGE
my-module-v3.7.4   Superseded                   5d23h
my-module-v3.7.5   Deployed                     5d23h

    Find the module release deployed in the cluster in the list (it should have the status Deployed).

  2. Add the annotation modules.deckhouse.io/reinstall=true to the expanded release:

    d8 k annotate mr my-module-v3.7.5 modules.deckhouse.io/reinstall=true

After adding the annotation, the module image is re-downloaded from the registry, the module is validated with the current settings from ModuleConfig, and installed in the cluster. After successful reinstallation, the annotation is automatically removed from ModuleRelease.

To verify that the module has been successfully reinstalled and all module pods are working, use the command:

d8 k -n d8-<module-name> get pods

All pods in the module must have the status Running. Example:

NAME                                READY   STATUS    RESTARTS   AGE
backend-567d6c6cdc-g5qgt            1/1     Running   0          2d2h
frontend-7c8b567759-h8jdf           1/1     Running   0          2d2h
observability-gw-86cf75f5d6-7xljh   1/1     Running   0          2d2h

Cluster and Infrastructure

What to do if you encounter problems updating Kubernetes components on cluster nodes, synchronizing nodes, or applying NodeGroup Configuration?

If Kubernetes components are not updated on the cluster node, the NodeGroup configuration is not applied, and not all NodeGroup nodes are synchronized (have the UPTODATE status), perform the following steps:

  1. Check the bashible logs on the node where the problems are occurring. The bashible mechanism is used to keep cluster nodes up to date. It is started by the bashible.timer timer at regular intervals as a service on the cluster nodes. This involves restarting, synchronizing scripts, and executing them (if necessary).

    To check bashible logs, use the command:

    journalctl -u bashible

    If the response contains the message Configuration is in sync, nothing to do, the node is synchronized and there are no problems. The absence of this message or the presence of errors indicates a problem.

  2. Check the synchronization status of cluster nodes using the command:

    d8 k get ng

    The number of nodes in the UPTODATE state must match the total number of nodes in each group.

    Example output:

    NAME       TYPE     READY   NODES   UPTODATE   INSTANCES   DESIRED   MIN   MAX   STANDBY   STATUS   AGE    SYNCED
frontend   Static   1       1       1                                                               118d   True
master     Static   3       3       3                                                               118d   True
system     Static   2       2       2                                                               118d   True
worker     Static   2       2       2                                                               118d   True

What to do if problems adding a node to the cluster via Cluster API Provider Static are present?

If, when adding a node to the cluster via Cluster API Provider Static (CAPS), it remains in Pending or Bootstrapping status, perform the following steps:

  1. Verify that the access keys specified in the SSHCredentials resource are correct. Ensure that the username and SSH key specified in SSHCredentials are correct.

  2. On the node where the problem occurred, check that the public key corresponding to the private key from SSHCredentials is present in authorized_keys. Example command for checking:

    cat ~/.ssh/authorized_keys

  3. Check the number of nodes specified in NodeGroup, which should include the node being added. Make sure that the maximum number of nodes is not exceeded.

  4. Check the status of the bashible.service on the node that caused the problem:

    systemctl status bashible.service

    It must have the status active (running). If the service has the status inactive or failed, the service has not started. This indicates a problem with the configuration process.

  5. If the steps above did not resolve the issue, remove the problematic node and its StaticInstance resource from the cluster so that the system will attempt to recreate them. To do this:

    • Get a list of nodes and locate the problematic one:

      d8 k get nodes

    • Find the corresponding StaticInstance resource:

      kubectl get staticinstances -n <namespace-name>

    • Remove the problematic node:

      kubectl delete node <node-name>

    • Remove the corresponding StaticInstance resource:

      kubectl delete staticinstances -n <namespace-name> <static-instance-name>

How to change the instance type for nodes with the CloudPermanent type?

To change the instance type for nodes with the CloudPermanent type, follow these steps:

  1. Make a backup of etcd and the /etc/kubernetes directory.
  2. Transfer the archive to a server outside the cluster (e.g., on a local machine).
  3. Ensure there are no alerts in the cluster that can prevent the update of the master nodes.

  4. Make sure that Deckhouse queue is empty. To view the status of all Deckhouse job queues, run the following command:

    d8 s queue list

    Example output (queues are empty):

    Summary:
- 'main' queue: empty.
- 88 other queues (0 active, 88 empty): 0 tasks.
- no tasks to handle.

  5. Run the appropriate edition and version of the Deckhouse installer container on the local machine (change the container registry address if necessary):

    DH_VERSION=$(d8 k -n d8-system get deployment deckhouse -o jsonpath='{.metadata.annotations.core\.deckhouse\.io\/version}') 
DH_EDITION=$(d8 k -n d8-system get deployment deckhouse -o jsonpath='{.metadata.annotations.core\.deckhouse\.io\/edition}' | tr '[:upper:]' '[:lower:]' ) 
docker run --pull=always -it -v "$HOME/.ssh/:/tmp/.ssh/" \
  registry.deckhouse.ru/deckhouse/${DH_EDITION}/install:${DH_VERSION} bash

  6. In the installer container, run the following command to check the state before working:

    dhctl terraform check --ssh-agent-private-keys=/tmp/.ssh/<SSH_KEY_FILENAME> --ssh-user=<USERNAME> \
  --ssh-host <MASTER-NODE-0-HOST> --ssh-host <MASTER-NODE-1-HOST> --ssh-host <MASTER-NODE-2-HOST>

    The command output should indicate that Terraform found no inconsistencies and no changes are required.

  7. In the installer container, run the command to edit the cluster configuration (specify the addresses of all master nodes in the --ssh-host parameter):

    dhctl config edit provider-cluster-configuration --ssh-agent-private-keys=/tmp/.ssh/<SSH_KEY_FILENAME> --ssh-user=<USERNAME> \
  --ssh-host <MASTER-NODE-0-HOST> --ssh-host <MASTER-NODE-1-HOST> --ssh-host <MASTER-NODE-2-HOST>

  8. Edit the instanceClass parameter of the desired node group by changing the instance type and save the changes. Example settings for the masterNodeGroup of the Yandex Cloud provider:

    masterNodeGroup:
 replicas: 3  # required number of master nodes
 instanceClass:
   cores: 4      # change the number of CPUs
   memory: 8192  # change the memory size (in MB)
   # other instance parameters...
   externalIPAddresses:
   - "Auto"      # for each master node
   - "Auto"
   - "Auto"

  9. In the installer container, run the following command to perform nodes upgrade:

    You should read carefully what converge is going to do when it asks for approval.

    When the command is executed, the nodes will be replaced by new nodes with confirmation on each node. The replacement will be performed one by one in reverse order (2,1,0).

    dhctl converge --ssh-agent-private-keys=/tmp/.ssh/<SSH_KEY_FILENAME> --ssh-user=<USERNAME> \
  --ssh-host <MASTER-NODE-0-HOST> --ssh-host <MASTER-NODE-1-HOST> --ssh-host <MASTER-NODE-2-HOST>

    Repeat the steps below (Sec. 9-12) for each master node one by one, starting with the node with the highest number (suffix 2) and ending with the node with the lowest number (suffix 0).

  10. On the newly created node, check the systemd-unit log for the bashible.service. Wait until the node configuration is complete (you will see a message nothing to do in the log):

    journalctl -fu bashible.service

  11. Make sure the node is listed as an etcd cluster member:

    for pod in $(d8 k -n kube-system get pod -l component=etcd,tier=control-plane -o name); do
  d8 k -n kube-system exec "$pod" -- etcdctl --cacert /etc/kubernetes/pki/etcd/ca.crt \
  --cert /etc/kubernetes/pki/etcd/ca.crt --key /etc/kubernetes/pki/etcd/ca.key \
  --endpoints https://127.0.0.1:2379/ member list -w table
  if [ $? -eq 0 ]; then
    break
  fi
done

  12. Make sure control-plane-manager is running on the node:

    d8 k -n kube-system wait pod --timeout=10m --for=condition=ContainersReady \
  -l app=d8-control-plane-manager --field-selector spec.nodeName=<MASTER-NODE-N-NAME>
  13. Proceed to update the next node (repeat the steps above).

What to do if it takes a long time to switch to custom nodes in lower-priority groups?

When using multiple node groups with different priorities in a cloud cluster (the spec.cloudInstances.priority parameter), switching to nodes from groups with lower priority can take a long time. An example of this scenario is when groups of preemptible nodes (spot, etc.) are set to the highest priority, and if such instances are unavailable, ordering nodes from other groups takes a very long time.

When provisioning nodes, the cluster autoscaler sequentially selects the group with the highest priority that is not in a backoff state. A backoff is a temporary lock on a group following a failed attempt to provision a node (for example, due to a lack of instances of the required type in the cloud).

How it works when using multiple node groups:

  1. The cluster autoscaler attempts to provision a node in the group with the highest priority.
  2. If the node is not provisioned within the time specified by the max-node-provision-time parameter, the attempt is considered a failure.
  3. The group is marked as failed and blocked for the duration specified in the initial-node-group-backoff-duration parameter.
  4. If the same group fails again, the lockout time is doubled, but does not exceed the time specified in the max-node-group-backoff-duration parameter.
  5. The cluster autoscaler selects the next highest-priority group that is not locked out.

Here:

Parameter Default value Description
initial-node-group-backoff-duration 5 minutes Initial duration of the group lockout after the first failure. Doubles after each failed node ordering attempt
max-node-group-backoff-duration 30 minutes Maximum duration of the group lockout
max-node-provision-time 15 minutes The time after which the cluster autoscaler considers a node provisioning attempt to have failed

If these parameters are set to their default values, switching to node ordering in lower-priority groups may take a considerable amount of time (for more details, see this example).

These settings can be adjusted to speed up the switching of nodes in lower-priority groups. To set the desired values for the initial-node-group-backoff-duration, max-node-group-backoff-duration, and max-node-provision-time parameters, edit the cluster-autoscaler Deployment object in the d8-cloud-instance-manager namespace. The parameters are specified in the args field of the cluster-autoscaler container. Example:

...
     containers:
      - name: cluster-autoscaler
        args:
        - --initial-node-group-backoff-duration=1m # The initial duration of the group lockout after the first failure has been reduced.
        - --max-node-provision-time=5m # The time after which the cluster autoscaler considers a node request to have failed has been reduced.
...

If you encounter issues with node ordering in a cluster with a single node group

If cloud cluster uses only one node group (for example, exclusively for ordering spot or preemptible nodes) and you encounter issues with ordering such nodes during automatic scaling, follow these steps:

  1. Create one or more InstanceClass objects to be used when creating instances in the cluster. In the InstanceClass, specify node types that differ from those listed in the group mentioned above.

    In DP, InstanceClass objects vary depending on the provider. Examples: AWSInstanceClass, AzureInstanceClass, YandexInstanceClass.

  2. Create one or more new node groups (NodeGroup). In the spec.cloudInstances.classReference parameter, specify the InstanceClass objects created in the previous step. In the spec.cloudInstances.priority parameter, set the priority of the node group.
  3. If necessary, modify the max-node-provision-time, max-node-group-backoff-duration, and initial-node-group-backoff-duration parameters to speed up switching to lower-priority groups.

Example of the node ordering process with default settings

Suppose there are 3 groups of nodes with different priorities:

  • Group A (priority 50)
  • Group B (priority 30)
  • Group C (priority 0)

In this case, the node ordering process might be as follows:

Time Event
10:00 The cluster autoscaler attempts to order a node in A
10:15 The attempt to provision a node in A within 15 minutes (max-node-provision-time) fails → A blocked for 5 minutes (initial-node-group-backoff-duration)
10:15:xx The cluster autoscaler attempts to order a node in B (next in priority, not blocked)
10:30 The attempt to provision a node in B within 15 minutes (max-node-provision-time) fails → B blocked for 5 minutes (initial-node-group-backoff-duration)
10:30:xx The cluster autoscaler attempts to order a node in A again, as the lockout period (the default value for initial-node-group-backoff-duration is 5 minutes) has ended
10:45 The attempt to order a node in A fails again → A is blocked for 10 minutes (initial-node-group-backoff-duration for A is doubled)
… And so on, until the lockout time for A and B exceeds 15 minutes (max-node-provision-time). In this case, the queue may not reach group C for about 1.5 hours

What to do if the resources generated by an object are not created after adding the object?

If the required resources do not appear after creating an object in the system (for example, dexAuthenticator), follow these steps:

  1. Check if there are any critical alerts in the cluster that may block the creation of the required objects. To do this, use the command:

    d8 k get clusteralerts.deckhouse.io

    Example output:

    NAME               ALERT                                           SEVERITY   AGE   LAST RECEIVED   STATUS
012f602592aa7a91   K8SSchedulerTargetDown                          3          16h   54s             firing
0836dc893d5ecc65   KubernetesDeploymentReplicasUnavailable         5          15h   62s             firing
08742f87d62d0063   NTPDaemonOnNodeDoesNotSynchronizeTime           5          16h   46s             firing
172cfd38d2f7fd19   D8DeckhouseQueueIsHung                          7          12h   66s             firing
1c5705daf731f5cf   D8StrongholdNoActiveNodes                       3          16h   55s             firing
1d2c2f7d69f69f4b   D8DeckhouseIsNotOnReleaseChannel                9          12h   53s             firing
205a551243d795f3   D8LogShipperAgentNotScheduledInCluster          7          15h   63s             firing
2e34039aa7a3018e   D8NodeIsNotUpdating                             9          12h   47s             firing
31baf9a70d657275   D8StrongholdClusterNotHealthy                   7          16h   55s             firing

    For more information about alerts, see the The list of alerts section.

  2. Check the Deckhouse task queue:

    d8 s queue list

    Example output (queues are empty):

    Summary:
- 'main' queue: empty.
- 88 other queues (0 active, 88 empty): 0 tasks.
- no tasks to handle.

    If there are many unprocessed or long-running tasks in the queue, this may indicate problems.

  3. Analyze DP logs and events:

    • To view logs in real time, use the command:

      d8 k -n d8-system logs -f -l app=deckhouse

      Example output:

      {"level":"info","logger":"addon-operator","msg":"ConvergeModules task for OperatorStartup in phase '', trigger is Operator-Startup","binding":"ConvergeModules","event.type":"OperatorStartup","queue":"main","task.flow":"start","task.id":"fde0eb3b-5c3e-4da6-a0d8-a52f8ae03428","time":"2025-11-26T08:29:33Z"}
{"level":"warn","logger":"addon-operator.converge-modules","msg":"ConvergeModules: functional scheduler not finished","binding":"ConvergeModules","event.type":"OperatorStartup","queue":"main","task.id":"fde0eb3b-5c3e-4da6-a0d8-a52f8ae03428","time":"2025-11-26T08:29:33Z"}

      When analyzing logs, pay special attention to warnings (WARNING) and error messages (ERROR).

    • To view events, use the command:

      d8 k -n d8-system get events

      Example output:

      LAST SEEN   TYPE      REASON              OBJECT                                          MESSAGE
11m         Warning   Unhealthy           pod/deckhouse-5886c9bd77-vgdbw                  Readiness probe failed: HTTP probe failed with statuscode: 500
7m22s       Normal    SuccessfulDelete    replicaset/deckhouse-5886c9bd77                 Deleted pod: deckhouse-5886c9bd77-vgdbw
7m20s       Normal    Scheduled           pod/deckhouse-6bc5c4494-fwx6z                   Successfully assigned d8-system/deckhouse-6bc5c4494-fwx6z to sandbox1-master-0
7m20s       Normal    Pulling             pod/deckhouse-6bc5c4494-fwx6z                   Pulling image "dev-registry.deckhouse.io/sys/deckhouse-oss@sha256:17ac07634e17422df52720264cddec3916ed6985a77782dc8a24fe5352290e6e"

    When analyzing events, pay special attention to those with the Warning type.

Deckhouse

How can I check DP versions available in different release channels?

For information about the current DP versions across all release channels, visit releases.deckhouse.io.

How to check the job queue in Deckhouse?

How to check the status of all Deckhouse task queues?

To view the status of all Deckhouse job queues, run the following command:

d8 s queue list

Example output (queues are empty):

Summary:
- 'main' queue: empty.
- 88 other queues (0 active, 88 empty): 0 tasks.
- no tasks to handle.

How to view the status of the main task queue?

To view the status of the Deckhouse main task queue, run the following command:

d8 s queue main

Example output (38 tasks in the main queue):

Queue 'main': length 38, status: 'run first task'

Example output (the main queue is empty):

Queue 'main': length 0, status: 'waiting for task 0s'

What should I do if DP is not receiving updates from the configured channel?

  • Ensure the correct release channel is configured.

  • Check that DNS resolution for the Deckhouse image registry is working correctly.

    Get and compare the IP addresses of registry.deckhouse.io from both a node and the deckhouse Pod. They must match.

    Example of obtaining an IP of registry.deckhouse.io from a node:

    getent ahosts registry.deckhouse.io

    Example output:

    185.193.90.38    STREAM registry.deckhouse.io
185.193.90.38    DGRAM
185.193.90.38    RAW

    Example of obtaining an IP of registry.deckhouse.io from the deckhouse Pod:

    d8 k -n d8-system exec -ti svc/deckhouse-leader -c deckhouse -- getent ahosts registry.deckhouse.io

    Example output:

    185.193.90.38    STREAM registry.deckhouse.io
185.193.90.38    DGRAM  registry.deckhouse.io

    If the resulted IPs do not match, check DNS settings on the node. Pay attention to the search domain list in /etc/resolv.conf, which affects name resolution in the deckhouse Pod. If the search parameter in /etc/resolv.conf file specifies a domain with wildcard DNS resolution configured, this may lead to incorrect IP address resolution for the Deckhouse image registry (see example below).

Example DNS settings that may cause issues resolving the DP image registry IP address

Below is an example of how DNS settings may result in different resolution behavior on the node and in a Kubernetes Pod:

  • Example of /etc/resolv.conf on the node:

    nameserver 10.0.0.10
search company.my

    On nodes, the default ndot setting is 1 (options ndots:1), while in Kubernetes Pods, it’s 5. This causes different resolution logic for DNS names with 5 or fewer dots on a node and on the Pod.

  • The DNS zone company.my has a wildcard entry *.company.my that resolves to 10.0.0.100. This means any undefined DNS name in the company.my zone resolves to 10.0.0.100.

Taking into account the search parameter in /etc/resolv.conf, when accessing registry.deckhouse.io from a node, the system will attempt to resolve the IP address for registry.deckhouse.io (because it considers it fully qualified due to the default options ndots:1 setting).

However, when accessing registry.deckhouse.io from a Kubernetes Pod, considering the options ndots:5 setting used by default in Kubernetes and the search parameter, the system will first attempt to resolve the name registry.deckhouse.io.company.my. This name will resolve to the IP address 10.0.0.100 because, according to the company.my DNS zone’s wildcard configuration, *.company.my is resolved to 10.0.0.100. As a result, the Pod will fail to connect to the registry.deckhouse.io host and will be unable to download information about available Deckhouse updates.

Does Deckhouse support realtime (rt) and low-latency Linux kernels?

In general, realtime (rt) and lowlatency kernels are supported with no extra configuration needed for Deckhouse Platform. Such kernels have been tested with DP EE version v1.75.7 and above on the following OSes and kernels:

  • CentOS 9 Stream, linux-5.14.0-706.el9.x86_64+rt
  • RedOS 8, linux-6.12.85-1.red80.x86_64-rt
  • Astra Linux 1.7.5, linux-5.15-lowlatency

How can I know when a new DP version is available for the cluster?

As soon as a new version appears on the configured release channel:

  • The DeckhouseReleaseIsWaitingManualApproval alert will appear if the cluster is in manual update mode.
  • A new DeckhouseRelease custom resource will be created. To see the list of releases, run d8 k get deckhousereleases. If the new version is in Pending state, it means it hasn’t been installed yet. Possible reasons:
    • Manual update mode is enabled.
    • Automatic update mode is enabled and update windows are scheduled, but the window hasn’t started yet.
    • Automatic update mode is enabled and update windows are not scheduled, but the update is delayed by a random period to reduce load on the container image registry. The status.message field of the DeckhouseRelease resource will show a corresponding message.
    • The update.notification.minimalNotificationTime parameter is set, and the delay period hasn’t elapsed.

How can I receive information about upcoming updates in advance?

You can get information about upcoming minor DP version updates on the release channel in one of the following ways:

How can I tell that the update was successful?

If the DeckhouseUpdating alert is gone, the update has finished.

You can also check the status of DP releases in the cluster with the following command:

d8 k get deckhouserelease

Example output:

NAME       PHASE        TRANSITIONTIME   MESSAGE
v1.46.8    Superseded   13d
v1.46.9    Superseded   11d
v1.47.0    Superseded   4h12m
v1.47.1    Deployed     4h12m

The Deployed status means the cluster has switched to the corresponding version, but it doesn’t guarantee that the update has been successful.

To ensure the update completed successfully, check the state of the deckhouse Pod with the following command:

d8 k -n d8-system get pods -l app=deckhouse

Example output:

NAME                   READY  STATUS   RESTARTS  AGE
deckhouse-7844b47bcd-qtbx9  1/1   Running  0       1d
  • If the Pod is Running and shows 1/1 under READY, it means the update completed successfully.
  • If the Pod is Running but shows 0/1 under READY, it means the update is still in progress. If it stays like this for more than 20–30 minutes, it may indicate a problem with DP that requires investigation.
  • If the Pod is not Running, it may indicate a problem with DP that requires investigation.

If something goes wrong

  • Check the logs using the following command:

    d8 k -n d8-system logs -f -l app=deckhouse | jq -Rr 'fromjson? | .msg'

  • Collect debug information and contact DP technical support.

How can I tell that an update is in progress?

During an update:

  • The DeckhouseUpdating alert is active.
  • The deckhouse Pod is not in the Ready state. If the Pod stays in a non-Ready state for a long time, it may indicate an issue with DP that requires investigation.

What to do if there are problems updating DP?

Deckhouse Platform update fails, one or more Deckhouse pods are in an unworkable state

If the Deckhouse Platform update fails, one or more Deckhouse pods in the d8-system namespace are in an unworkable state. Perform the following steps:

  1. Check the Deckhouse logs using the command:

    d8 k -n d8-system logs -f -l app=deckhouse | jq -Rr 'fromjson? | .msg'

    If there are any problems, information about them will be included in the output. When analyzing logs, pay special attention to warnings (WARNING) and error messages (ERROR).

  2. Check Deckhouse events using the command:

    d8 k -n d8-system describe po -l app=deckhouse | awk '
/^Name:/ { 
    pod = $2; 
    print "=== " pod " ==="; 
    in_events = 0 
}
/Events:/ { 
    in_events = 1; 
    next 
}
in_events && /^$/ { 
    in_events = 0; 
    print "---" 
}
in_events && !/^Events:/ { 
    print $0 
}
' | sed '/^---$/N;/^\n$/D'

    Pod events contain key information about problems (e.g., planning errors, image loading errors, etc.). When analyzing events, pay special attention to those with the Warning type.

    Example output:

    Type     Reason     Age                      From     Message
----     ------     ----                     ----     -------
Warning  Unhealthy  4m44s (x1918 over 154m)  kubelet  Readiness probe failed: HTTP probe failed with statuscode: 500

DP update is stuck in the Release is suspended status

The status Release is suspended indicates that it has been postponed and is currently unavailable (not recommended) for installation. In this case, it is recommended to remain on the latest available release or on the one currently installed (it will have the status Deployed).

To view the list of releases, use the command:

d8 k get deckhousereleases.deckhouse.io

Example output:

NAME       PHASE        TRANSITIONTIME   MESSAGE
v1.69.13   Skipped      3h46m
v1.69.14   Skipped      3h46m
v1.69.15   Skipped      3h46m
v1.69.16   Superseded   160m
v1.70.12   Suspended    49d              Release is suspended
v1.70.13   Skipped      36d
v1.70.14   Skipped      34d
v1.70.15   Skipped      28d
v1.70.16   Skipped      19d
v1.70.17   Deployed     160m
v1.71.3    Suspended    14d              Release is suspended

IAM

What to do if there are problems applying DexProvider settings?

If you have changed the DexProvider settings in the user-authn module and one of the following problems occurs:

  • No changes are visible (settings are not applied).
  • When attempting to log in to the platform’s web interface with any type of authorization, a 500 Internal Server Error occurs without a detailed description.

Perform the following steps:

  1. Check the status of the dex deployment:

    d8 k -n d8-user-authn get pod

    Example output:

    NAME                                    READY   STATUS    RESTARTS   AGE
dex-5ddb779b7d-6pbhs                    2/2     Running   0          20h
kubeconfig-generator-7c46977b9f-5kdmc   1/1     Running   0          20h

    If the module is functioning properly and the correct configuration is specified in DexProvider, all pods will have the status Running.

  2. Check the logs for the problematic pod:

    d8 k -n d8-user-authn logs dex-<pod-name>

    Based on the information from the logs, correct the configuration in the DexProvider resource and wait for the dex pods to restart. Within a few minutes, the pods will restart automatically, and the platform’s web interface (located at console.<CLUSTER_NAME_TEMPLATE>) will become available and will reflect the changes made to the DexProvider resource.

Kubernetes and Scheduling

What to do when the API server is overloaded?

The following signs may indicate problems with API server load and memory consumption:

  • kubectl (d8) responds slowly or does not respond at all (commands are executed slowly or not at all).
  • Pods are recreated in the cluster for no apparent reason.

If these signs are present, perform the following actions:

  1. Check the resource consumption of API server pods. To do this, use the command:

    d8 k -n kube-system top po -l component=kube-apiserver

    Pay attention to MEMORY consumption and CPU.

    Example output:

    NAME                               CPU(cores)   MEMORY(bytes)
kube-apiserver-sandbox1-master-0   251m         1476Mi

  2. Check the metrics in Grafana.

    To view the metrics, open the dashboard “Home” → “Dashboards” → “Kubernetes Cluster” → “Control Plane Status”. Review the graphs related to the API server (“Kube-apiserver CPU Usage”, “Kube-apiserver Memory Usage”, “Kube-apiserver latency”, etc.).

  3. Review the API server audit logs to identify the source of high memory consumption. One common cause of high memory consumption is a large number of requests.

How to check the Kubernetes version in use?

To check the Kubernetes version in use, run the command:

d8 k get nodes

Example output:

NAME                   STATUS   ROLES                  AGE    VERSION
frontend-0             Ready    frontend               118d   v1.31.9
master-0               Ready    control-plane,master   118d   v1.31.9
master-1               Ready    control-plane,master   118d   v1.31.9
master-2               Ready    control-plane,master   118d   v1.31.9
system-0               Ready    system                 118d   v1.31.9
system-1               Ready    system                 118d   v1.31.9
worker-0               Ready    worker                 37d    v1.31.9
worker-1               Ready    worker                 19d    v1.31.9

Network

Is the ingress-nginx module supported? Will it affect passing a PCI DSS audit?

The ingress-nginx module remains supported by Deckhouse Platform for the entire platform support lifecycle and does not depend on the upstream project’s status. The Deckhouse team tracks vulnerabilities in the controller and related components (NGINX, Lua modules, base images), delivers fixes in DP releases, and provides a migration path toward Gateway API.

Flant is the vendor responsible for support. Together with ongoing module maintenance and vulnerability management, this allows passing a PCI DSS audit and meets expectations for vendor accountability for a platform component.

Virtualization

How to use Ansible to provision virtual machines?

Ansible is an automation tool for running tasks on remote servers over SSH. This example shows how to use Ansible with virtual machines (VMs) in the demo-app project.

The example assumes that:

  • demo-app namespace contains a VM named frontend.
  • VM has a cloud user with SSH access.
  • Private SSH key on the machine where Ansible runs is stored in /home/user/.ssh/id_rsa.

  1. Create an inventory.yaml file:

    ---
all:
  vars:
    ansible_ssh_common_args: '-o ProxyCommand="d8 v port-forward --stdio=true %h %p"'
    # Default user for SSH access.
    ansible_user: cloud
    # Path to private key.
    ansible_ssh_private_key_file: /home/user/.ssh/id_rsa
  hosts:
    # Host name in the format <VM_NAME>.<NAMESPACE>.
    frontend.demo-app:

  2. Check the virtual machine uptime:

    ansible -m shell -a "uptime" -i inventory.yaml all

# frontend.demo-app | CHANGED | rc=0 >>
# 12:01:20 up 2 days,  4:59,  0 users,  load average: 0.00, 0.00, 0.00

If you do not want to use an inventory file, pass all parameters on the command line:

ansible -m shell -a "uptime" \
  -i "frontend.demo-app," \
  -e "ansible_ssh_common_args='-o ProxyCommand=\"d8 v port-forward --stdio=true %h %p\"'" \
  -e "ansible_user=cloud" \
  -e "ansible_ssh_private_key_file=/home/user/.ssh/id_rsa" \
  all

How to automatically generate inventory for Ansible?

The d8 v ansible-inventory command requires d8 v0.27.0 or higher.

The command works only for virtual machines that have the main cluster network (Main) connected.

Instead of manually creating an inventory file, you can use the d8 v ansible-inventory command, which automatically generates an Ansible inventory from virtual machines in the specified namespace. The command is compatible with the ansible inventory script interface.

Only machines in the Running phase that have an assigned IP address get into the inventory. Host names are formatted as <VM_NAME>.<NAMESPACE> (for example, frontend.demo-app).

  1. Optionally set host variables via annotations (for example, the SSH user):

    d8 k -n demo-app annotate vm frontend vars.ansible.deckhouse.io/ansible_user="cloud"

  2. Run Ansible with a dynamically generated inventory:

    ANSIBLE_INVENTORY_ENABLED=yaml ansible -m shell -a "uptime" all -i <(d8 v ansible-inventory -n demo-app -o yaml)

The <(...) construct is necessary because Ansible expects a file or script as the source of the host list. Simply specifying the command in quotes won’t work, because Ansible tries to execute the string as a script. The <(...) construct passes the command output as a file that Ansible can read.

  1. Or save the inventory to a file and run the check:

    d8 v ansible-inventory --list -o yaml -n demo-app > inventory.yaml
ansible -m shell -a "uptime" -i inventory.yaml all

How to use cloud-init to configure virtual machines?

Cloud-init is used for initial guest OS configuration on first boot. The configuration is written in YAML and starts with the #cloud-config directive.

When using cloud images (for example, official distribution images), you must provide a cloud-init configuration. Without it, some distributions do not configure network connectivity, and the virtual machine becomes unreachable on the network, even if the main network (Main) is attached.

In addition, cloud images do not allow login by default — you must either add SSH keys for the default user or create a new user with SSH access. Otherwise, you will not be able to access the virtual machine.

Updating and installing packages

Example cloud-config for updating the system and installing packages from a list:

#cloud-config
# Update package lists.
package_update: true
# Upgrade installed packages to latest versions.
package_upgrade: true
# List of packages to install.
packages:
  - nginx
  - curl
  - htop
# Commands to run after package installation.
runcmd:
  - systemctl enable --now nginx.service

Creating a user

Example cloud-config for creating a local user with a password and SSH key:

#cloud-config
# List of users to create.
users:
    # Username.
  - name: cloud
    # Password hash.
    passwd: "<PASSWORD_HASH>"
    # Do not lock the account.
    lock_passwd: false
    # Sudo privileges without a password prompt.
    sudo: ALL=(ALL) NOPASSWD:ALL
    # Default shell.
    shell: /bin/bash
    # SSH keys for access.
    ssh-authorized-keys:
      - <SSH_PUBLIC_KEY>
# Allow password authentication via SSH.
ssh_pwauth: true

To generate a password hash for the passwd field, run:

mkpasswd --method=SHA-512 --rounds=4096

Creating a file with required permissions

Example cloud-config for creating a file with specified access permissions:

#cloud-config
# List of files to create.
write_files:
    # File path.
  - path: /opt/scripts/start.sh
    # File content.
    content: |
      #!/bin/bash
      echo "Starting application"
    # File owner, user and group.
    owner: cloud:cloud
    # Access permissions in octal format.
    permissions: '0755'

Configuring disk and filesystem

Example cloud-config for disk partitioning, filesystem creation, and mounting:

#cloud-config
# Disk partitioning setup.
disk_setup:
  # Disk device.
  /dev/sdb:
    # Partition table type, gpt or mbr.
    table_type: gpt
    # Automatically create partitions.
    layout: true
    # Do not overwrite existing partitions.
    overwrite: false

# Filesystem setup.
fs_setup:
    # Filesystem label.
  - label: data
    # Filesystem type.
    filesystem: ext4
    # Partition device.
    device: /dev/sdb1
    # Automatically detect partition.
    partition: auto

# Filesystem mounting.
mounts:
  # [device, mount_point, fs_type, options, dump, pass]
  - ["/dev/sdb1", "/mnt/data", "ext4", "defaults", "0", "2"]

Configuring network interfaces for additional networks

The settings described in this section apply only to additional networks. The main network (Main) is configured automatically via cloud-init and does not require manual configuration.

Additional networks are configured manually via cloud-init. The write_files block creates the configuration files, and the runcmd block applies the settings.

Connecting additional networks to a virtual machine is covered in Additional network interfaces.

The following examples cover common ways to configure networking in the guest OS:

  • systemd-networkd
  • Netplan (Ubuntu)
  • ifcfg (RHEL/CentOS)
  • Alpine Linux

Example cloud-config for distributions that use systemd-networkd (Debian, CoreOS, and others):

#cloud-config
write_files:
  - path: /etc/systemd/network/10-eth1.network
    content: |
      [Match]
      Name=eth1

      [Network]
      Address=192.168.1.10/24
      Gateway=192.168.1.1
      DNS=8.8.8.8

runcmd:
  - systemctl restart systemd-networkd

Example cloud-config for Ubuntu and other systems that use Netplan:

#cloud-config
write_files:
  - path: /etc/netplan/99-custom.yaml
    content: |
      network:
        version: 2
        ethernets:
          eth1:
            addresses:
              - 10.0.0.5/24
            gateway4: 10.0.0.1
            nameservers:
              addresses: [8.8.8.8]
          eth2:
            dhcp4: true

runcmd:
  - netplan apply

Example cloud-config for RHEL-compatible distributions that use the ifcfg scheme and NetworkManager:

#cloud-config
write_files:
  - path: /etc/sysconfig/network-scripts/ifcfg-eth1
    content: |
      DEVICE=eth1
      BOOTPROTO=none
      ONBOOT=yes
      IPADDR=192.168.1.10
      PREFIX=24
      GATEWAY=192.168.1.1
      DNS1=8.8.8.8

runcmd:
  - nmcli connection reload
  - nmcli connection up eth1

Example cloud-config for distributions that use the traditional /etc/network/interfaces format (Alpine and similar):

#cloud-config
write_files:
  - path: /etc/network/interfaces
    append: true
    content: |
      auto eth1
      iface eth1 inet static
          address 192.168.1.10
          netmask 255.255.255.0
          gateway 192.168.1.1

runcmd:
  - /etc/init.d/networking restart

How to increase the DVCR size?

The DVCR volume size is set in the virtualization module ModuleConfig (spec.settings.dvcr.storage.persistentVolumeClaim.size). The new value must be greater than the current one.

  1. Check the current DVCR size:

    d8 k get mc virtualization -o jsonpath='{.spec.settings.dvcr.storage.persistentVolumeClaim}'

    Example output:

    {"size":"58G","storageClass":"linstor-thick-data-r1"}

  2. Increase size using patch (set the value you need):

    d8 k patch mc virtualization \
  --type merge -p '{"spec": {"settings": {"dvcr": {"storage": {"persistentVolumeClaim": {"size":"59G"}}}}}}'

    Example output:

    moduleconfig.deckhouse.io/virtualization patched

  3. Verify that ModuleConfig shows the new size:

    d8 k get mc virtualization -o jsonpath='{.spec.settings.dvcr.storage.persistentVolumeClaim}'

    Example output:

    {"size":"59G","storageClass":"linstor-thick-data-r1"}

  4. Check the current DVCR status:

    d8 k get pvc dvcr -n d8-virtualization

    Example output:

    NAME STATUS VOLUME                                    CAPACITY    ACCESS MODES   STORAGECLASS           AGE
dvcr Bound  pvc-6a6cedb8-1292-4440-b789-5cc9d15bbc6b  57617188Ki  RWO            linstor-thick-data-r1  7d

How to change the DVCR StorageClass when a PVC already exists?

You can change the DVCR storage StorageClass only by recreating the PVC. All images previously loaded into DVCR are lost, along with data for existing ClusterVirtualImage and VirtualImage resources.

The spec.settings.dvcr.storage.persistentVolumeClaim.storageClassName field in the virtualization module ModuleConfig sets the StorageClass for the virtual machine image storage volume (DVCR). While a PVC for that volume exists in the d8-virtualization namespace, you cannot change the field via the API.

You cannot change storageClassName on an existing PVC in place, and DVCR data is not migrated between storage classes.

To change the DVCR StorageClass, perform the following steps:

  1. Stop DVCR:

    d8 k -n d8-virtualization scale deployment dvcr --replicas=0

  2. List PVCs in the d8-virtualization namespace and find the PVC for the DVCR volume:

    d8 k get pvc -n d8-virtualization

  3. Delete the PVC you found. Replace <PVC_NAME> with the resource name. If the command fails because of insufficient permissions, run it as system:sudouser:

    d8 k --as system:sudouser -n d8-virtualization delete pvc/<PVC_NAME>

  4. Set the new StorageClass in ModuleConfig, substituting the class you need for <STORAGE_CLASS_NAME>:

    d8 k patch mc virtualization --type merge -p '{"spec":{"settings":{"dvcr":{"storage":{"persistentVolumeClaim":{"storageClassName":"<STORAGE_CLASS_NAME>"}}}}}}'

    Example output:

    moduleconfig.deckhouse.io/virtualization patched

  5. Start DVCR:

    d8 k -n d8-virtualization scale deployment dvcr --replicas=1

  6. Verify the PVC:

    d8 k get pvc -n d8-virtualization

    Example output:

    NAME   STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          VOLUMEATTRIBUTESCLASS   AGE
dvcr   Bound    pvc-b43f2e33-32cc-435a-aa1d-b53df35b030a   100Gi      RWO            linstor-thin-r1-hdd   <unset>                 34s

The storage for the chosen StorageClass must be reachable from the nodes where DVCR runs: system nodes, or worker nodes if the cluster has no system nodes.

How to install an operating system in a virtual machine from an ISO image?

Below is a typical Windows guest OS installation scenario from an ISO image. Before you begin, host the ISO on an HTTP endpoint reachable from the cluster.

  1. Create an empty VirtualDisk for OS installation:

    apiVersion: virtualization.deckhouse.io/v1alpha2
kind: VirtualDisk
metadata:
  name: win-disk
  namespace: default
spec:
  persistentVolumeClaim:
    size: 100Gi
    storageClassName: local-path

  2. Create ClusterVirtualImage resources for the Windows OS ISO and the VirtIO driver ISO:

    apiVersion: virtualization.deckhouse.io/v1alpha2
kind: ClusterVirtualImage
metadata:
  name: win-11-iso
spec:
  dataSource:
    type: HTTP
    http:
      url: "http://example.com/win11.iso"

    apiVersion: virtualization.deckhouse.io/v1alpha2
kind: ClusterVirtualImage
metadata:
  name: win-virtio-iso
spec:
  dataSource:
    type: HTTP
    http:
      url: "https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso"

  3. Create a virtual machine (VM):

    apiVersion: virtualization.deckhouse.io/v1alpha2
kind: VirtualMachine
metadata:
  name: win-vm
  namespace: default
  labels:
    vm: win
spec:
  virtualMachineClassName: generic
  runPolicy: Manual
  osType: Windows
  bootloader: EFI
  cpu:
    cores: 6
    coreFraction: 50%
  memory:
    size: 8Gi
  enableParavirtualization: true
  blockDeviceRefs:
    - kind: VirtualDisk
      name: win-disk
    - kind: ClusterVirtualImage
      name: win-11-iso
    - kind: ClusterVirtualImage
      name: win-virtio-iso

  4. Start the virtual machine:

    d8 v start win-vm

  5. Connect to the VM console and complete the OS installation and VirtIO drivers using the graphical installer.

    VNC connection:

    d8 v vnc -n default win-vm

  6. After the installation is complete, restart the virtual machine.

  7. For further work, connect via VNC again:

    d8 v vnc -n default win-vm

How to create a golden image for Linux?

A golden image is a pre-configured virtual machine (VM) image that can be used to quickly create new VMs with pre-installed software and settings.

  1. Create a virtual machine, install the required software on it, and perform all necessary configurations.

  2. Install and configure qemu-guest-agent (recommended):

    • For RHEL/CentOS:

      yum install -y qemu-guest-agent

    • For Debian/Ubuntu:

      apt-get update
apt-get install -y qemu-guest-agent

  3. Enable and start the service:

    systemctl enable qemu-guest-agent
systemctl start qemu-guest-agent

  4. Set the machine run policy to AlwaysOnUnlessStoppedManually, otherwise you will not be able to shut it down.

  5. Prepare the image. Clean unused filesystem blocks:

    fstrim -v /
fstrim -v /boot

  6. Clean network settings:

    • For RHEL:

      nmcli con delete $(nmcli -t -f NAME,DEVICE con show | grep -v ^lo: | cut -d: -f1)
rm -f /etc/sysconfig/network-scripts/ifcfg-eth*

    • For Debian/Ubuntu:

      rm -f /etc/network/interfaces.d/*

  7. Clean system identifiers:

    echo -n > /etc/machine-id
rm -f /var/lib/dbus/machine-id
ln -s /etc/machine-id /var/lib/dbus/machine-id

  8. Remove the SSH host keys:

    rm -f /etc/ssh/ssh_host_*

  9. Clean the systemd journal:

    journalctl --vacuum-size=100M --vacuum-time=7d

  10. Clean package manager cache:

    • For RHEL:

      yum clean all

    • For Debian/Ubuntu:

      apt-get clean

  11. Clean temporary files:

    rm -rf /tmp/*
rm -rf /var/tmp/*

  12. Clean logs:

    find /var/log -name "*.log" -type f -exec truncate -s 0 {} \;

  13. Clean command history:

    history -c

  14. On RHEL, reset and restore the SELinux contexts in one of two ways.

    Restore the contexts right away:

    restorecon -R /

    Or schedule a relabel for the next boot:

    touch /.autorelabel

  15. Verify that /etc/fstab references UUID or LABEL rather than names like /dev/sdX:

    blkid
cat /etc/fstab

  16. Reset cloud-init state (logs and seed):

    cloud-init clean --logs --seed

  17. Perform final synchronization and buffer cleanup:

    sync
echo 3 > /proc/sys/vm/drop_caches

  18. Shut down the virtual machine:

    poweroff

  19. Create a VirtualImage resource that references the prepared VM’s VirtualDisk:

    d8 k apply -f -<<EOF
apiVersion: virtualization.deckhouse.io/v1alpha2
kind: VirtualImage
metadata:
  name: <IMAGE_NAME>
  namespace: <NAMESPACE>
spec:
  dataSource:
    type: ObjectRef
    objectRef:
      kind: VirtualDisk
      name: <SOURCE_DISK_NAME>
EOF

    Or create a ClusterVirtualImage resource so the image is available cluster-wide for all projects:

    d8 k apply -f -<<EOF
apiVersion: virtualization.deckhouse.io/v1alpha2
kind: ClusterVirtualImage
metadata:
  name: <IMAGE_NAME>
spec:
  dataSource:
    type: ObjectRef
    objectRef:
      kind: VirtualDisk
      name: <SOURCE_DISK_NAME>
      namespace: <NAMESPACE>
EOF

    Here, <IMAGE_NAME> is the name of the image being created, <NAMESPACE> is the namespace of the prepared machine, and <SOURCE_DISK_NAME> is the name of its disk.

  20. Create a new VirtualDisk from the resulting image:

    d8 k apply -f -<<EOF
apiVersion: virtualization.deckhouse.io/v1alpha2
kind: VirtualDisk
metadata:
  name: <VM_DISK_NAME>
  namespace: <NAMESPACE>
spec:
  dataSource:
    type: ObjectRef
    objectRef:
      kind: VirtualImage
      name: <IMAGE_NAME>
EOF

    Here, <VM_DISK_NAME> is the name of the disk of the new machine.

After completing these steps, you will have a golden image that can be used to quickly create new virtual machines with pre-installed software and configurations.

How to restore the cluster if images from registry.deckhouse.io cannot be pulled after a license change?

After a license change on a cluster with containerd v1 and removal of the outdated license, images from registry.deckhouse.io may stop being pulled. Nodes then retain the outdated configuration file /etc/containerd/conf.d/dvcr.toml, which is not removed automatically. Because of it, the registry module does not start, and without it DVCR does not work.

Applying a NodeGroupConfiguration (NGC) manifest removes the file on the nodes. After the registry module starts, delete the manifest, since this is a one-time fix.

  1. Save the manifest to a file (for example, containerd-dvcr-remove-old-config.yaml):

    apiVersion: deckhouse.io/v1alpha1
kind: NodeGroupConfiguration
metadata:
  name: containerd-dvcr-remove-old-config.sh
spec:
  weight: 32 # Must be in range 32–90.
  nodeGroups: ["*"]
  bundles: ["*"]
  content: |
    # Copyright 2023 Flant JSC
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #      http://www.apache.org/licenses/LICENSE-2.0
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.

    rm -f /etc/containerd/conf.d/dvcr.toml

  2. Apply the saved manifest:

    d8 k apply -f containerd-dvcr-remove-old-config.yaml

  3. Verify that the registry module is running:

    d8 k -n d8-system -o yaml get secret registry-state | yq -C -P '.data | del .state | map_values(@base64d) | .conditions = (.conditions | from_yaml)'

    Example output when the registry module has started successfully:

    conditions:
# ...
  - lastTransitionTime: "..."
    message: ""
    reason: ""
    status: "True"
    type: Ready

  4. Delete the one-time NodeGroupConfiguration manifest:

    d8 k delete -f containerd-dvcr-remove-old-config.yaml

The migration procedure is covered in Migrating container runtime to containerd v2.

How to redirect traffic to a virtual machine?

A virtual machine runs in a Kubernetes cluster, so traffic reaches it the same way it reaches any other workload. Routing is handled by the standard Kubernetes Service resource, which selects targets by labels.

  1. Create a service with the required settings.

    For example, consider a virtual machine with the label vm: frontend-0, an HTTP service exposed on ports 80 and 443, and SSH access on port 22:

    apiVersion: virtualization.deckhouse.io/v1alpha2
kind: VirtualMachine
metadata:
  name: frontend-0
  namespace: dev
  labels:
    vm: frontend-0
spec: ...

  2. To route network traffic to the virtual machine’s ports, create a service:

    This service listens on ports 80 and 443 and forwards traffic to the corresponding ports of the target virtual machine. SSH access from outside is provided on port 2211:

    apiVersion: v1
kind: Service
metadata:
  name: frontend-0-svc
  namespace: dev
spec:
  type: LoadBalancer
  ports:
  - name: ssh
    port: 2211
    protocol: TCP
    targetPort: 22
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    vm: frontend-0

Why is only one connection method available for a virtual machine?

You can connect to a virtual machine (VM) via the serial console (d8 v console) or VNC (d8 v vnc). These methods use different communication channels with the guest OS and depend on its configuration. Both methods are covered in Connecting to a virtual machine.

The sections below describe common situations where only one connection method works.

Why does VNC not work when the serial console is available?

VNC displays the guest OS screen and requires virtual terminal support in the kernel. The serial console works independently of the graphics subsystem.

Check in the guest OS whether virtual terminal support is enabled in the kernel configuration:

cat /boot/config-$(uname -r) | grep CONFIG_VT

The output should show CONFIG_VT=y:

CONFIG_VT=y

If the output shows CONFIG_VT is not set, rebuild the kernel with the option enabled or use an OS image with a suitable kernel configuration.

Why does the serial console not work when VNC is available?

The serial console connects to the ttyS0 port in the guest OS. If the getty service for this port is not running, d8 v console will not show a login prompt even though VNC continues to work.

In the guest OS, enable and start the serial-getty service for ttyS0:

sudo systemctl enable --now serial-getty@ttyS0.service

Then connect to the serial console again.

How to provide a Windows answer file (Sysprep)?

Unattended Windows installation uses an answer file (unattend.xml or autounattend.xml).

The example answer file below:

  • Sets the English UI language and keyboard layout.
  • Connects the VirtIO drivers for the setup stage (the order of devices in blockDeviceRefs on the VirtualMachine resource must match the paths in the file).
  • Creates disk layout for installation with EFI.
  • Creates the cloud administrator and the regular user account.
<p><b>Example of the contents of the autounattend.xml file…</b>&lt;/summary&gt;</p>

<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?xml version="1.0" encoding="utf-8"?&gt;</span> <span class="nt">&lt;unattend</span> <span class="na">xmlns=</span><span class="s">"urn:schemas-microsoft-com:unattend"</span> <span class="na">xmlns:wcm=</span><span class="s">"http://schemas.microsoft.com/WMIConfig/2002/State"</span><span class="nt">&gt;</span>   <span class="nt">&lt;settings</span> <span class="na">pass=</span><span class="s">"offlineServicing"</span><span class="nt">&gt;&lt;/settings&gt;</span>   <span class="nt">&lt;settings</span> <span class="na">pass=</span><span class="s">"windowsPE"</span><span class="nt">&gt;</span>
<span class="nt">&lt;component</span> <span class="na">name=</span><span class="s">"Microsoft-Windows-International-Core-WinPE"</span> <span class="na">processorArchitecture=</span><span class="s">"amd64"</span> <span class="na">publicKeyToken=</span><span class="s">"31bf3856ad364e35"</span> <span class="na">language=</span><span class="s">"neutral"</span> <span class="na">versionScope=</span><span class="s">"nonSxS"</span><span class="nt">&gt;</span>
  <span class="nt">&lt;SetupUILanguage&gt;</span>
    <span class="nt">&lt;UILanguage&gt;</span>en-US<span class="nt">&lt;/UILanguage&gt;</span>
  <span class="nt">&lt;/SetupUILanguage&gt;</span>
  <span class="nt">&lt;InputLocale&gt;</span>0409:00000409<span class="nt">&lt;/InputLocale&gt;</span>
  <span class="nt">&lt;SystemLocale&gt;</span>en-US<span class="nt">&lt;/SystemLocale&gt;</span>
  <span class="nt">&lt;UILanguage&gt;</span>en-US<span class="nt">&lt;/UILanguage&gt;</span>
  <span class="nt">&lt;UserLocale&gt;</span>en-US<span class="nt">&lt;/UserLocale&gt;</span>
<span class="nt">&lt;/component&gt;</span>
<span class="nt">&lt;component</span> <span class="na">name=</span><span class="s">"Microsoft-Windows-PnpCustomizationsWinPE"</span> <span class="na">processorArchitecture=</span><span class="s">"amd64"</span> <span class="na">publicKeyToken=</span><span class="s">"31bf3856ad364e35"</span> <span class="na">language=</span><span class="s">"neutral"</span> <span class="na">versionScope=</span><span class="s">"nonSxS"</span><span class="nt">&gt;</span>
  <span class="nt">&lt;DriverPaths&gt;</span>
    <span class="nt">&lt;PathAndCredentials</span> <span class="na">wcm:keyValue=</span><span class="s">"4b29ba63"</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
      <span class="nt">&lt;Path&gt;</span>E:\amd64\w11<span class="nt">&lt;/Path&gt;</span>
    <span class="nt">&lt;/PathAndCredentials&gt;</span>
    <span class="nt">&lt;PathAndCredentials</span> <span class="na">wcm:keyValue=</span><span class="s">"25fe51ea"</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
      <span class="nt">&lt;Path&gt;</span>E:\NetKVM\w11\amd64<span class="nt">&lt;/Path&gt;</span>
    <span class="nt">&lt;/PathAndCredentials&gt;</span>
  <span class="nt">&lt;/DriverPaths&gt;</span>
<span class="nt">&lt;/component&gt;</span>
<span class="nt">&lt;component</span> <span class="na">name=</span><span class="s">"Microsoft-Windows-Setup"</span> <span class="na">processorArchitecture=</span><span class="s">"amd64"</span> <span class="na">publicKeyToken=</span><span class="s">"31bf3856ad364e35"</span> <span class="na">language=</span><span class="s">"neutral"</span> <span class="na">versionScope=</span><span class="s">"nonSxS"</span><span class="nt">&gt;</span>
  <span class="nt">&lt;DiskConfiguration&gt;</span>
    <span class="nt">&lt;Disk</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
      <span class="nt">&lt;DiskID&gt;</span>0<span class="nt">&lt;/DiskID&gt;</span>
      <span class="nt">&lt;WillWipeDisk&gt;</span>true<span class="nt">&lt;/WillWipeDisk&gt;</span>
      <span class="nt">&lt;CreatePartitions&gt;</span>
        <span class="c">&lt;!-- Recovery partition --&gt;</span>
        <span class="nt">&lt;CreatePartition</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
          <span class="nt">&lt;Order&gt;</span>1<span class="nt">&lt;/Order&gt;</span>
          <span class="nt">&lt;Type&gt;</span>Primary<span class="nt">&lt;/Type&gt;</span>
          <span class="nt">&lt;Size&gt;</span>250<span class="nt">&lt;/Size&gt;</span>
        <span class="nt">&lt;/CreatePartition&gt;</span>
        <span class="c">&lt;!-- EFI system partition (ESP) --&gt;</span>
        <span class="nt">&lt;CreatePartition</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
          <span class="nt">&lt;Order&gt;</span>2<span class="nt">&lt;/Order&gt;</span>
          <span class="nt">&lt;Type&gt;</span>EFI<span class="nt">&lt;/Type&gt;</span>
          <span class="nt">&lt;Size&gt;</span>100<span class="nt">&lt;/Size&gt;</span>
        <span class="nt">&lt;/CreatePartition&gt;</span>
        <span class="c">&lt;!-- Microsoft reserved partition (MSR) --&gt;</span>
        <span class="nt">&lt;CreatePartition</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
          <span class="nt">&lt;Order&gt;</span>3<span class="nt">&lt;/Order&gt;</span>
          <span class="nt">&lt;Type&gt;</span>MSR<span class="nt">&lt;/Type&gt;</span>
          <span class="nt">&lt;Size&gt;</span>128<span class="nt">&lt;/Size&gt;</span>
        <span class="nt">&lt;/CreatePartition&gt;</span>
        <span class="c">&lt;!-- Windows partition --&gt;</span>
        <span class="nt">&lt;CreatePartition</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
          <span class="nt">&lt;Order&gt;</span>4<span class="nt">&lt;/Order&gt;</span>
          <span class="nt">&lt;Type&gt;</span>Primary<span class="nt">&lt;/Type&gt;</span>
          <span class="nt">&lt;Extend&gt;</span>true<span class="nt">&lt;/Extend&gt;</span>
        <span class="nt">&lt;/CreatePartition&gt;</span>
      <span class="nt">&lt;/CreatePartitions&gt;</span>
      <span class="nt">&lt;ModifyPartitions&gt;</span>
        <span class="c">&lt;!-- Recovery partition --&gt;</span>
        <span class="nt">&lt;ModifyPartition</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
          <span class="nt">&lt;Order&gt;</span>1<span class="nt">&lt;/Order&gt;</span>
          <span class="nt">&lt;PartitionID&gt;</span>1<span class="nt">&lt;/PartitionID&gt;</span>
          <span class="nt">&lt;Label&gt;</span>Recovery<span class="nt">&lt;/Label&gt;</span>
          <span class="nt">&lt;Format&gt;</span>NTFS<span class="nt">&lt;/Format&gt;</span>
          <span class="nt">&lt;TypeID&gt;</span>de94bba4-06d1-4d40-a16a-bfd50179d6ac<span class="nt">&lt;/TypeID&gt;</span>
        <span class="nt">&lt;/ModifyPartition&gt;</span>
        <span class="c">&lt;!-- EFI system partition (ESP) --&gt;</span>
        <span class="nt">&lt;ModifyPartition</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
          <span class="nt">&lt;Order&gt;</span>2<span class="nt">&lt;/Order&gt;</span>
          <span class="nt">&lt;PartitionID&gt;</span>2<span class="nt">&lt;/PartitionID&gt;</span>
          <span class="nt">&lt;Label&gt;</span>System<span class="nt">&lt;/Label&gt;</span>
          <span class="nt">&lt;Format&gt;</span>FAT32<span class="nt">&lt;/Format&gt;</span>
        <span class="nt">&lt;/ModifyPartition&gt;</span>
        <span class="c">&lt;!-- MSR partition does not need to be modified --&gt;</span>
        <span class="c">&lt;!-- Windows partition --&gt;</span>
        <span class="nt">&lt;ModifyPartition</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
          <span class="nt">&lt;Order&gt;</span>3<span class="nt">&lt;/Order&gt;</span>
          <span class="nt">&lt;PartitionID&gt;</span>4<span class="nt">&lt;/PartitionID&gt;</span>
          <span class="nt">&lt;Label&gt;</span>Windows<span class="nt">&lt;/Label&gt;</span>
          <span class="nt">&lt;Letter&gt;</span>C<span class="nt">&lt;/Letter&gt;</span>
          <span class="nt">&lt;Format&gt;</span>NTFS<span class="nt">&lt;/Format&gt;</span>
        <span class="nt">&lt;/ModifyPartition&gt;</span>
      <span class="nt">&lt;/ModifyPartitions&gt;</span>
    <span class="nt">&lt;/Disk&gt;</span>
    <span class="nt">&lt;WillShowUI&gt;</span>OnError<span class="nt">&lt;/WillShowUI&gt;</span>
  <span class="nt">&lt;/DiskConfiguration&gt;</span>
  <span class="nt">&lt;ImageInstall&gt;</span>
    <span class="nt">&lt;OSImage&gt;</span>
      <span class="nt">&lt;InstallTo&gt;</span>
        <span class="nt">&lt;DiskID&gt;</span>0<span class="nt">&lt;/DiskID&gt;</span>
        <span class="nt">&lt;PartitionID&gt;</span>4<span class="nt">&lt;/PartitionID&gt;</span>
      <span class="nt">&lt;/InstallTo&gt;</span>
    <span class="nt">&lt;/OSImage&gt;</span>
  <span class="nt">&lt;/ImageInstall&gt;</span>
  <span class="nt">&lt;UserData&gt;</span>
    <span class="nt">&lt;ProductKey&gt;</span>
      <span class="nt">&lt;Key&gt;&lt;PRODUCT_KEY&gt;&lt;/Key&gt;</span>
      <span class="nt">&lt;WillShowUI&gt;</span>OnError<span class="nt">&lt;/WillShowUI&gt;</span>
    <span class="nt">&lt;/ProductKey&gt;</span>
    <span class="nt">&lt;AcceptEula&gt;</span>true<span class="nt">&lt;/AcceptEula&gt;</span>
  <span class="nt">&lt;/UserData&gt;</span>
  <span class="nt">&lt;UseConfigurationSet&gt;</span>false<span class="nt">&lt;/UseConfigurationSet&gt;</span>
<span class="nt">&lt;/component&gt;</span>   <span class="nt">&lt;/settings&gt;</span>   <span class="nt">&lt;settings</span> <span class="na">pass=</span><span class="s">"generalize"</span><span class="nt">&gt;&lt;/settings&gt;</span>   <span class="nt">&lt;settings</span> <span class="na">pass=</span><span class="s">"specialize"</span><span class="nt">&gt;</span>
<span class="nt">&lt;component</span> <span class="na">name=</span><span class="s">"Microsoft-Windows-Deployment"</span> <span class="na">processorArchitecture=</span><span class="s">"amd64"</span> <span class="na">publicKeyToken=</span><span class="s">"31bf3856ad364e35"</span> <span class="na">language=</span><span class="s">"neutral"</span> <span class="na">versionScope=</span><span class="s">"nonSxS"</span><span class="nt">&gt;</span>
  <span class="nt">&lt;RunSynchronous&gt;</span>
    <span class="nt">&lt;RunSynchronousCommand</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
      <span class="nt">&lt;Order&gt;</span>1<span class="nt">&lt;/Order&gt;</span>
      <span class="nt">&lt;Path&gt;</span>powershell.exe -NoProfile -Command "$xml = [xml]::new(); $xml.Load('C:\Windows\Panther\unattend.xml'); $sb = [scriptblock]::Create( $xml.unattend.Extensions.ExtractScript ); Invoke-Command -ScriptBlock $sb -ArgumentList $xml;"<span class="nt">&lt;/Path&gt;</span>
    <span class="nt">&lt;/RunSynchronousCommand&gt;</span>
    <span class="nt">&lt;RunSynchronousCommand</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
      <span class="nt">&lt;Order&gt;</span>2<span class="nt">&lt;/Order&gt;</span>
      <span class="nt">&lt;Path&gt;</span>powershell.exe -NoProfile -Command "Get-Content -LiteralPath 'C:\Windows\Setup\Scripts\Specialize.ps1' -Raw | Invoke-Expression;"<span class="nt">&lt;/Path&gt;</span>
    <span class="nt">&lt;/RunSynchronousCommand&gt;</span>
    <span class="nt">&lt;RunSynchronousCommand</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
      <span class="nt">&lt;Order&gt;</span>3<span class="nt">&lt;/Order&gt;</span>
      <span class="nt">&lt;Path&gt;</span>reg.exe load "HKU\DefaultUser" "C:\Users\Default\NTUSER.DAT"<span class="nt">&lt;/Path&gt;</span>
    <span class="nt">&lt;/RunSynchronousCommand&gt;</span>
    <span class="nt">&lt;RunSynchronousCommand</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
      <span class="nt">&lt;Order&gt;</span>4<span class="nt">&lt;/Order&gt;</span>
      <span class="nt">&lt;Path&gt;</span>powershell.exe -NoProfile -Command "Get-Content -LiteralPath 'C:\Windows\Setup\Scripts\DefaultUser.ps1' -Raw | Invoke-Expression;"<span class="nt">&lt;/Path&gt;</span>
    <span class="nt">&lt;/RunSynchronousCommand&gt;</span>
    <span class="nt">&lt;RunSynchronousCommand</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
      <span class="nt">&lt;Order&gt;</span>5<span class="nt">&lt;/Order&gt;</span>
      <span class="nt">&lt;Path&gt;</span>reg.exe unload "HKU\DefaultUser"<span class="nt">&lt;/Path&gt;</span>
    <span class="nt">&lt;/RunSynchronousCommand&gt;</span>
  <span class="nt">&lt;/RunSynchronous&gt;</span>
<span class="nt">&lt;/component&gt;</span>   <span class="nt">&lt;/settings&gt;</span>   <span class="nt">&lt;settings</span> <span class="na">pass=</span><span class="s">"auditSystem"</span><span class="nt">&gt;&lt;/settings&gt;</span>   <span class="nt">&lt;settings</span> <span class="na">pass=</span><span class="s">"auditUser"</span><span class="nt">&gt;&lt;/settings&gt;</span>   <span class="nt">&lt;settings</span> <span class="na">pass=</span><span class="s">"oobeSystem"</span><span class="nt">&gt;</span>
<span class="nt">&lt;component</span> <span class="na">name=</span><span class="s">"Microsoft-Windows-International-Core"</span> <span class="na">processorArchitecture=</span><span class="s">"amd64"</span> <span class="na">publicKeyToken=</span><span class="s">"31bf3856ad364e35"</span> <span class="na">language=</span><span class="s">"neutral"</span> <span class="na">versionScope=</span><span class="s">"nonSxS"</span><span class="nt">&gt;</span>
  <span class="nt">&lt;InputLocale&gt;</span>0409:00000409<span class="nt">&lt;/InputLocale&gt;</span>
  <span class="nt">&lt;SystemLocale&gt;</span>en-US<span class="nt">&lt;/SystemLocale&gt;</span>
  <span class="nt">&lt;UILanguage&gt;</span>en-US<span class="nt">&lt;/UILanguage&gt;</span>
  <span class="nt">&lt;UserLocale&gt;</span>en-US<span class="nt">&lt;/UserLocale&gt;</span>
<span class="nt">&lt;/component&gt;</span>
<span class="nt">&lt;component</span> <span class="na">name=</span><span class="s">"Microsoft-Windows-Shell-Setup"</span> <span class="na">processorArchitecture=</span><span class="s">"amd64"</span> <span class="na">publicKeyToken=</span><span class="s">"31bf3856ad364e35"</span> <span class="na">language=</span><span class="s">"neutral"</span> <span class="na">versionScope=</span><span class="s">"nonSxS"</span><span class="nt">&gt;</span>
  <span class="nt">&lt;UserAccounts&gt;</span>
    <span class="nt">&lt;LocalAccounts&gt;</span>
      <span class="nt">&lt;LocalAccount</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
        <span class="nt">&lt;Name&gt;</span>cloud<span class="nt">&lt;/Name&gt;</span>
        <span class="nt">&lt;DisplayName&gt;</span>cloud<span class="nt">&lt;/DisplayName&gt;</span>
        <span class="nt">&lt;Group&gt;</span>Administrators<span class="nt">&lt;/Group&gt;</span>
        <span class="nt">&lt;Password&gt;</span>
          <span class="nt">&lt;Value&gt;&lt;ADMIN_PASSWORD&gt;&lt;/Value&gt;</span>
          <span class="nt">&lt;PlainText&gt;</span>true<span class="nt">&lt;/PlainText&gt;</span>
        <span class="nt">&lt;/Password&gt;</span>
      <span class="nt">&lt;/LocalAccount&gt;</span>
      <span class="nt">&lt;LocalAccount</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
        <span class="nt">&lt;Name&gt;</span>User<span class="nt">&lt;/Name&gt;</span>
        <span class="nt">&lt;DisplayName&gt;</span>user<span class="nt">&lt;/DisplayName&gt;</span>
        <span class="nt">&lt;Group&gt;</span>Users<span class="nt">&lt;/Group&gt;</span>
        <span class="nt">&lt;Password&gt;</span>
          <span class="nt">&lt;Value&gt;&lt;USER_PASSWORD&gt;&lt;/Value&gt;</span>
          <span class="nt">&lt;PlainText&gt;</span>true<span class="nt">&lt;/PlainText&gt;</span>
        <span class="nt">&lt;/Password&gt;</span>
      <span class="nt">&lt;/LocalAccount&gt;</span>
    <span class="nt">&lt;/LocalAccounts&gt;</span>
  <span class="nt">&lt;/UserAccounts&gt;</span>
  <span class="nt">&lt;AutoLogon&gt;</span>
    <span class="nt">&lt;Username&gt;</span>cloud<span class="nt">&lt;/Username&gt;</span>
    <span class="nt">&lt;Enabled&gt;</span>true<span class="nt">&lt;/Enabled&gt;</span>
    <span class="nt">&lt;LogonCount&gt;</span>1<span class="nt">&lt;/LogonCount&gt;</span>
    <span class="nt">&lt;Password&gt;</span>
      <span class="nt">&lt;Value&gt;&lt;ADMIN_PASSWORD&gt;&lt;/Value&gt;</span>
      <span class="nt">&lt;PlainText&gt;</span>true<span class="nt">&lt;/PlainText&gt;</span>
    <span class="nt">&lt;/Password&gt;</span>
  <span class="nt">&lt;/AutoLogon&gt;</span>
  <span class="nt">&lt;OOBE&gt;</span>
    <span class="nt">&lt;ProtectYourPC&gt;</span>3<span class="nt">&lt;/ProtectYourPC&gt;</span>
    <span class="nt">&lt;HideEULAPage&gt;</span>true<span class="nt">&lt;/HideEULAPage&gt;</span>
    <span class="nt">&lt;HideWirelessSetupInOOBE&gt;</span>true<span class="nt">&lt;/HideWirelessSetupInOOBE&gt;</span>
    <span class="nt">&lt;HideOnlineAccountScreens&gt;</span>false<span class="nt">&lt;/HideOnlineAccountScreens&gt;</span>
  <span class="nt">&lt;/OOBE&gt;</span>
  <span class="nt">&lt;FirstLogonCommands&gt;</span>
    <span class="nt">&lt;SynchronousCommand</span> <span class="na">wcm:action=</span><span class="s">"add"</span><span class="nt">&gt;</span>
      <span class="nt">&lt;Order&gt;</span>1<span class="nt">&lt;/Order&gt;</span>
      <span class="nt">&lt;CommandLine&gt;</span>powershell.exe -NoProfile -Command "Get-Content -LiteralPath 'C:\Windows\Setup\Scripts\FirstLogon.ps1' -Raw | Invoke-Expression;"<span class="nt">&lt;/CommandLine&gt;</span>
    <span class="nt">&lt;/SynchronousCommand&gt;</span>
  <span class="nt">&lt;/FirstLogonCommands&gt;</span>
<span class="nt">&lt;/component&gt;</span>   <span class="nt">&lt;/settings&gt;</span> <span class="nt">&lt;/unattend&gt;</span> </code></pre></div>    </div>

<p>&lt;/details&gt;</p>

<p>Replace <code class="language-plaintext highlighter-rouge">&lt;PRODUCT_KEY&gt;</code> with your Windows product key, and <code class="language-plaintext highlighter-rouge">&lt;ADMIN_PASSWORD&gt;</code> and <code class="language-plaintext highlighter-rouge">&lt;USER_PASSWORD&gt;</code> with the passwords of the accounts being created. Windows reads these passwords from the file in plain text, so don’t leave the finished answer file on a shared resource.</p>

<ol>
  <li>
    <p>Save the answer file as <code class="language-plaintext highlighter-rouge">autounattend.xml</code> (use the example above or adjust it to your needs).</p>
  </li>
  <li>
    <p>Create a secret with the type <code class="language-plaintext highlighter-rouge">provisioning.virtualization.deckhouse.io/sysprep</code>:</p>

    <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>d8 k create secret generic sysprep-config <span class="nt">--type</span><span class="o">=</span><span class="s2">"provisioning.virtualization.deckhouse.io/sysprep"</span> <span class="nt">--from-file</span><span class="o">=</span>./autounattend.xml </code></pre></div>        </div>
  </li>
  <li>
    <p>Create a virtual machine that will use the answer file during installation. Specify <code class="language-plaintext highlighter-rouge">provisioning</code> with type <code class="language-plaintext highlighter-rouge">SysprepRef</code> in the specification. If necessary, add other Base64-encoded files to the specification required for the answer file scripts to run successfully.</p>

    <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">virtualization.deckhouse.io/v1alpha2</span> <span class="na">kind</span><span class="pi">:</span> <span class="s">VirtualMachine</span> <span class="na">metadata</span><span class="pi">:</span>   <span class="na">name</span><span class="pi">:</span> <span class="s">win-vm</span>   <span class="na">namespace</span><span class="pi">:</span> <span class="s">default</span>   <span class="na">labels</span><span class="pi">:</span>
<span class="na">vm</span><span class="pi">:</span> <span class="s">win</span> <span class="na">spec</span><span class="pi">:</span>   <span class="na">virtualMachineClassName</span><span class="pi">:</span> <span class="s">generic</span>   <span class="na">provisioning</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">SysprepRef</span>
<span class="na">sysprepRef</span><span class="pi">:</span>
  <span class="na">kind</span><span class="pi">:</span> <span class="s">Secret</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">sysprep-config</span>   <span class="na">runPolicy</span><span class="pi">:</span> <span class="s">AlwaysOn</span>   <span class="na">osType</span><span class="pi">:</span> <span class="s">Windows</span>   <span class="na">bootloader</span><span class="pi">:</span> <span class="s">EFI</span>   <span class="na">cpu</span><span class="pi">:</span>
<span class="na">cores</span><span class="pi">:</span> <span class="m">6</span>
<span class="na">coreFraction</span><span class="pi">:</span> <span class="s">50%</span>   <span class="na">memory</span><span class="pi">:</span>
<span class="na">size</span><span class="pi">:</span> <span class="s">8Gi</span>   <span class="na">enableParavirtualization</span><span class="pi">:</span> <span class="kc">true</span>   <span class="na">blockDeviceRefs</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">kind</span><span class="pi">:</span> <span class="s">VirtualDisk</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">win-disk</span>
<span class="pi">-</span> <span class="na">kind</span><span class="pi">:</span> <span class="s">ClusterVirtualImage</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">win-11-iso</span>
<span class="pi">-</span> <span class="na">kind</span><span class="pi">:</span> <span class="s">ClusterVirtualImage</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">win-virtio-iso</span> </code></pre></div>        </div>
  </li>
</ol>