Cloud Hypervisor学习笔记 – 快速开始

Docs: 下载

Wed, 15 Oct 2025 00:00:00 +0000

下载

github 下载地址:

https://github.com/cloud-hypervisor/cloud-hypervisor/releases/

x86

x86平台下载:

mkdir -p ~/work/soft/cloudhypervisor/bin
cd ~/work/soft/cloudhypervisor/bin
wget https://github.com/cloud-hypervisor/cloud-hypervisor/releases/download/v50.0/ch-remote
wget https://github.com/cloud-hypervisor/cloud-hypervisor/releases/download/v50.0/cloud-hypervisor
wget https://github.com/cloud-hypervisor/cloud-hypervisor/releases/download/v50.0/ch-remote-static
wget https://github.com/cloud-hypervisor/cloud-hypervisor/releases/download/v50.0/cloud-hypervisor-static

chmod +x ./ch-remote
chmod +x ./cloud-hypervisor
chmod +x ./ch-remote-static
chmod +x ./cloud-hypervisor-static

# 验证一下版本
./cloud-hypervisor --version
./cloud-hypervisor-static --version
./ch-remote --version
./ch-remote-static --version

版本输出为:

$ ./cloud-hypervisor-static --version
cloud-hypervisor v50.0.0
$ ./cloud-hypervisor --version
[1]    10062 segmentation fault  ./cloud-hypervisor --version
$ ./ch-remote --version
ch-remote v50.0.0
$ ./ch-remote-static --version
ch-remote v50.0.0

cloud-hypervisor 不能用.这是因为带 static 的二进制文件是静态编译版本，所有依赖库都已经打包在可执行文件里；不带 static 的则是动态链接版本，运行时需要系统上存在相应的共享库。如果系统缺少或版本不匹配，就可能出现 segmentation fault。

方便起见,还是直接用 static 版本比较好.

mkdir -p ~/work/soft/cloudhypervisor/bin
cd ~/work/soft/cloudhypervisor/bin
wget https://github.com/cloud-hypervisor/cloud-hypervisor/releases/download/v50.0/ch-remote-static -O ch-remote
wget https://github.com/cloud-hypervisor/cloud-hypervisor/releases/download/v50.0/cloud-hypervisor-static -O cloud-hypervisor

chmod +x ./ch-remote
chmod +x ./cloud-hypervisor

# 验证一下版本
./cloud-hypervisor --version
./ch-remote --version

arm64

类似,下载 cloud-hypervisor-static-aarch64 和 ch-remote-static-aarch64．

安装

将二进制文件所在的目录加入到 path 就可以了,这样用起来比较方便.

vi ~/.zshrc

加入内容:

# cloud hypervisor
export PATH="$PATH:$HOME/work/soft/cloudhypervisor/bin"

生效并检验:

source ~/.zshrc

ch-remote --version
cloud-hypervisor --version

升级

貌似只要直接覆盖原来版本的文件即可．

Docs: 构建

Wed, 15 Oct 2025 00:00:00 +0000

也可以不下载,自己从源码开始构建.

参考: https://github.com/cloud-hypervisor/cloud-hypervisor/blob/main/docs/building.md

准备

机器信息:

操作系统: debian13
内核: 6.12

准备工作:

mkdir -p $HOME/work/soft/cloudhypervisor

export CLOUDH=$HOME/work/soft/cloudhypervisor

cd $CLOUDH

安装依赖包:

sudo apt-get update
sudo apt install git build-essential m4 bison flex uuid-dev qemu-utils musl-tools

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

rustup target add x86_64-unknown-linux-musl

构建

clone 代码并开始构建:

mkdir -p ~/work/code/cloud-hypervisor
cd ~/work/code/cloud-hypervisor

git clone https://github.com/cloud-hypervisor/cloud-hypervisor.git
cd cloud-hypervisor
# 最新的 release tag
git checkout v50.0
# 采用静态链接
cargo build --release --target=x86_64-unknown-linux-musl --all
sudo setcap cap_net_admin+ep ./target/x86_64-unknown-linux-musl/release/cloud-hypervisor

检查构建之后的版本：

./target/x86_64-unknown-linux-musl/release/cloud-hypervisor --version
cloud-hypervisor v50.0

ls -lh ./target/x86_64-unknown-linux-musl/release/cloud-hypervisor 
-rwxrwxr-x 2 sky sky 4.0M Nov 18 15:23 ./target/x86_64-unknown-linux-musl/release/cloud-hypervisor

静态链接也才4M,没大多少.

Docs: 启动

Wed, 15 Oct 2025 00:00:00 +0000

参考:

https://www.cloudhypervisor.org/docs/prologue/quick-start/#booting-linux

在 linux 上启动.

Cloud Hypervisor 支持直接内核启动（x86-64内核需使用支持PVH构建的内核）或通过固件/firmware启动（可选Rust Hypervisor固件或名为 CLOUDHV/CLOUDHV_EFI 的edk2 UEFI固件）。

最新版本的Rust Hypervisor固件及edk2存储库中均提供固件文件的二进制构建版本。

固件的选择取决于您的客户操作系统选项，可能需要进行一些实验性测试。

固件启动

Cloud Hypervisor 支持引导包含运行云工作负载所需全部组件的磁盘镜像，即云镜像。

准备工作

首先安装 qemu-utils,稍后要用到其中的 qemu-img 和 mkdosfs :

sudo apt install qemu-utils
sudo apt install dosfstools
sudo apt install mtools

以下示例命令将下载Ubuntu云镜像，将其转换为 Cloud Hypervisor 可用的格式，并生成引导该镜像所需的固件。

mkdir -p ~/work/soft/cloudhypervisor/fireware
cd ~/work/soft/cloudhypervisor/fireware

wget https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img

qemu-img convert -p -f qcow2 -O raw focal-server-cloudimg-amd64.img focal-server-cloudimg-amd64.raw

wget https://github.com/cloud-hypervisor/rust-hypervisor-firmware/releases/download/0.5.0/hypervisor-fw

处理完成后的文件列表:

$ ls -lh    

total 2.1G
-rw-rw-r-- 1 sky sky 618M Jun 25 09:39 focal-server-cloudimg-amd64.img
-rw-r--r-- 1 sky sky 2.2G Oct 16 16:54 focal-server-cloudimg-amd64.raw
-rw-rw-r-- 1 sky sky 133K Nov 30  2024 hypervisor-fw

Ubuntu云镜像默认不提供密码，因此首次启动时必须使用 cloud-init 磁盘镜像进行镜像定制。本脚本可生成基础 cloud-init 镜像，为镜像预设默认用户名/密码为 cloud/cloud123 。仅需在首次启动时添加此磁盘镜像即可。脚本同时通过 --net “mac=12:34:56:78:90:ab,tap=” 选项，基于 test_data/cloud-init/ubuntu/local/network-config 配置信息分配默认IP地址。随后系统将根据 network-config 详情启用匹配MAC地址的网络接口。

准备需要的脚本, 在代码仓库根目录下的 scripts 中:

https://github.com/cloud-hypervisor/cloud-hypervisor/tree/v48.0/scripts

后面还要使用到 test_data/cloud-init/ubuntu/ 目录下的文件, 因此最好是把整个代码仓库下载下来,方便后面使用

mkdir -p ~/work/code/cloud-hypervisor/
cd ~/work/code/cloud-hypervisor/

git clone https://github.com/cloud-hypervisor/cloud-hypervisor.git
cd cloud-hypervisor
git checkout v50.0

执行:

~/work/code/cloud-hypervisor/cloud-hypervisor/scripts/create-cloud-init.sh

输出如下:

+ rm -f /tmp/ubuntu-cloudinit.img
+ mkdosfs -n CIDATA -C /tmp/ubuntu-cloudinit.img 8192
mkfs.fat 4.2 (2021-01-31)
+ mcopy -oi /tmp/ubuntu-cloudinit.img -s test_data/cloud-init/ubuntu/local/user-data ::
+ mcopy -oi /tmp/ubuntu-cloudinit.img -s test_data/cloud-init/ubuntu/local/meta-data ::
+ mcopy -oi /tmp/ubuntu-cloudinit.img -s test_data/cloud-init/ubuntu/local/network-config ::

这会生成一个 ubuntu-cloudinit.img 文件,大小为 8M:

$ ls -lh /tmp/ubuntu-cloudinit.img
-rw-rw-r-- 1 sky sky 8.0M Oct 16 17:12 /tmp/ubuntu-cloudinit.img

注意这个目录是 /tmp,重启之后文件很可能就不在了,方便起见, 还是复制到

mkdir -p ~/work/soft/cloudhypervisor/images
mv /tmp/ubuntu-cloudinit.img /home/sky/work/soft/cloudhypervisor/images

启动

注意: 从 v49 版本开始需要设置 cloud-hypervisor 二进制的特权, 执行命令:

sudo setcap cap_net_admin+ep /home/sky/work/code/cloud-hypervisor/cloud-hypervisor/target/x86_64-unknown-linux-musl/release/cloud-hypervisor

否则后面启动时会报错:

cloud-hypervisor:   0.016376s: <main> ERROR:/home/sky/work/code/cloud-hypervisor/cloud-hypervisor/src/lib.rs:23 -- Fatal error: VmBoot(VmBoot(DeviceManager(CreateVirtioNet(OpenTap(TapOpen(ConfigureTap(Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" })))))))
Error: Cloud Hypervisor exited with the following chain of errors:
  0: Error booting VM
  1: The VM could not boot
  2: Error from device manager
  3: Cannot create virtio-net device
  4: Failed to open taps
  5: Open tap device failed
  6: Unable to configure tap interface
  7: Operation not permitted (os error 1)

注意启动命令中的路径要用绝对路径,不能用 ‘~/’ 之类,否则会报错无法找到文件.

/home/sky/work/code/cloud-hypervisor/cloud-hypervisor/target/x86_64-unknown-linux-musl/release/cloud-hypervisor \
	--kernel /home/sky/work/soft/cloudhypervisor/fireware/hypervisor-fw \
	--disk path=/home/sky/work/soft/cloudhypervisor/fireware/focal-server-cloudimg-amd64.raw path=/home/sky/work/soft/cloudhypervisor/images/ubuntu-cloudinit.img \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=,mac=,ip=,mask="

输出为:

cloud-hypervisor: 481.142000µs: <main> WARN:vmm/src/vm_config.rs:356 -- Deprecation warning: No IP address provided. A default IP address is assigned. This behavior will be deprecated soon.
cloud-hypervisor: 506.863000µs: <main> WARN:vmm/src/vm_config.rs:363 -- Deprecation warning: No network mask provided. A default network mask is assigned. This behavior will be deprecated soon.

Ubuntu 20.04.6 LTS cloud hvc0

cloud login:

输入前面设置的用户/密码 (cloud / cloud123), 登录进去, 如果要退出,执行 sudo shutdown now 命令关闭启动的 microVM 即可.

此时因为没有配置好网络, 所以登录进去 microVM 后会发现没有网络可用.

配置microVM的网络

没成功,等下继续.

首先要在主机上创建并配置 TAP 接口:

sudo ip tuntap add name vm-tap0 mode tap user $(whoami) 
sudo ip link set vm-tap0 up
sudo ip addr add 192.168.1.127/24 dev vm-tap0

此时在主机上看到的 vm-tap0 设备为:

$ ip addr
......

10: vm-tap0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 3e:7b:2e:ca:9a:3e brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.127/24 scope global vm-tap0
       valid_lft forever preferred_lft forever

注意：上面的 ip 命令在重启后会失效。若需持久化配置，可将其写入主机的网络配置文件（如 /etc/network/interfaces 或使用 systemd-networkd/NetworkManager)。

注意2: 如果配置有问题,可以先删除,再重新配置. sudo ip link delete vm-tap0

/home/sky/work/code/cloud-hypervisor/cloud-hypervisor/target/x86_64-unknown-linux-musl/release/cloud-hypervisor \
	--kernel /home/sky/work/soft/cloudhypervisor/fireware/hypervisor-fw \
	--disk path=/home/sky/work/soft/cloudhypervisor/fireware/focal-server-cloudimg-amd64.raw path=/tmp/ubuntu-cloudinit.img \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=vm-tap0,mac=12:34:56:78:90:22,ip=192.168.1.129,mask=255.255.255.0"


/home/sky/work/code/cloud-hypervisor/cloud-hypervisor/target/x86_64-unknown-linux-musl/release/cloud-hypervisor \
	--kernel /home/sky/work/soft/cloudhypervisor/fireware/hypervisor-fw \
	--disk path=/home/sky/work/soft/cloudhypervisor/fireware/focal-server-cloudimg-amd64.raw path=/home/sky/work/soft/cloudhypervisor/images/ubuntu-cloudinit.img \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=tap0,mac=12:34:56:78:90:22,ip=192.168.1.10,mask=255.255.255.0"

镜像启动

Cloud Hypervisor 也支持直接内核启动。对于 x86-64 架构，需要一个 vmlinux ELF 内核（编译时包含 PVH 支持）。为了支持开发，有一个自定义分支；不过只要所需选项已启用，任何近期的内核都可以使用。

构建 linux 内核

要构建内核：

cd ~/work/code/cloud-hypervisor/

git clone --depth 1 https://github.com/cloud-hypervisor/linux.git -b ch-6.12.8 linux-cloud-hypervisor
cd linux-cloud-hypervisor
make ch_defconfig

sudo apt install libelf-dev
sudo apt install bc

注意要执行 sudo apt install libelf-dev 命令安装 libelf,否则会报错:

<stdin>:1:10: fatal error: libelf.h: No such file or directory

构建 x86-64 kernel:

KCFLAGS="-Wa,-mx86-used-note=no" make bzImage -j `nproc`

构建完成后,linux 内核保存在 ./arch/x86/boot/compressed/vmlinux.bin,大小为 45M:

$ ls -lh ./arch/x86/boot/compressed/vmlinux.bin
-rwxrwxr-x 1 sky sky 45M Nov 18 15:50 ./arch/x86/boot/compressed/vmlinux.bin

rootfs 磁盘镜像可以用之前下载的 ubuntu 镜像, 之前存放在目录 ~/work/soft/cloudhypervisor/fireware 下

也可以再次下载和转换:

cd ~/work/soft/cloudhypervisor/fireware

wget https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img # x86-64
wget https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-arm64.img # AArch64
qemu-img convert -p -f qcow2 -O raw focal-server-cloudimg-amd64.img focal-server-cloudimg-amd64.raw # x86-64
qemu-img convert -p -f qcow2 -O raw focal-server-cloudimg-arm64.img focal-server-cloudimg-arm64.raw # AArch64

启动 vm:

/home/sky/work/code/cloud-hypervisor/cloud-hypervisor/target/x86_64-unknown-linux-musl/release/cloud-hypervisor \
	--kernel /home/sky/work/code/cloud-hypervisor/linux-cloud-hypervisor/arch/x86/boot/compressed/vmlinux.bin \
	--console off \
	--serial tty \
	--disk path=/home/sky/work/soft/cloudhypervisor/fireware/focal-server-cloudimg-amd64.raw \
	--cmdline "console=ttyS0 root=/dev/vda1 rw" \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=,mac=,ip=,mask="

Docs: 命令

Sat, 15 Nov 2025 00:00:00 +0000

参考:

https://www.cloudhypervisor.org/docs/prologue/commands/

cloud-hypervisor

通过 help 命令可以得到参数和参数说明:

cloud-hypervisor --help

--api-socket <api-socket>

HTTP API socket (UNIX domain socket): path=</path/to/a/file> or fd=<fd>
--balloon <balloon>

Balloon parameters

“size=<balloon_size>,deflate_on_oom=on|off”
--cmdline <cmdline>

Kernel command line
--console <console>

Control (virtio) console: “off|null|pty|tty|file=</path/to/a/file>,iommu=on|off”

[default: tty]
--cpus <cpus>

boot=<boot_vcpus>,max=<max_vcpus>,topology=<threads_per_core>:<cores_per_die>:<dies_per_package>:<packages>,kvm_hyperv=on

[default: boot=1,max_phys_bits=46]
--debug-console <debug-console>

off|pty|tty|file=</path/to/a/file>,iobase=<port in hex>

[default:off,iobase=0xe9]
--device <device>...

Direct device assignment parameters

“path=<device_path>,iommu=on|off,id=<device_id>,pci_segment=<segment_id>”
--disk <disk>...

Disk parameters

“path=<disk_image_path>,readonly=on|off,direct=on|off,iommu=on|off,num_queues=<number_of_queues>,queue_size=<size_of_each_queue>,vhost_user=on|off,socket=<vhost_user_socket_path>,bw_size=,bw_one_time_burst=,bw_refill_time=,ops_size=<io_ops>,ops_one_time_burst=<io_ops>,ops_refill_time=,id=<device_id>,pci_segment=<segment_id>,rate_limit_group=<group_id>,queue_affinity=<list_of_queue_indices_with_their_associated_cpuset>,serial=<serial_number>”
--event-monitor <event-monitor>

File to report events on: path=</path/to/a/file> or fd=<fd>
--firmware <firmware>

File to report events on: path=</path/to/a/file> or fd=<fd>
--fs <fs>...

virtio-fs parameters “tag=<tag_name>,socket=<socket_path>,num_queues=<number_of_queues>,queue_size=<size_of_each_queue>,id=<device_id>,pci_segment=<segment_id>”
--initramfs <initramfs>

Path to initramfs image
--kernel <kernel>

Path to kernel to load. This may be a kernel or firmware that supports a PVH entry point (e.g. vmlinux) or architecture equivalent
--landlock

enable/disable Landlock.
--landlock-rules <landlock-rules>...

Landlock parameters “path=<path/to/{file/dir}>,access=[rw]”
--log-file <log-file>

Log file. Standard error is used if not specified
--memory <memory>

“size=<guest_memory_size>,mergeable=on|off,shared=on|off,hugepages=on|off,hugepage_size=<hugepage_size>,hotplug_method=acpi|virtio-mem,hotplug_size=<hotpluggable_memory_size>,hotplugged_size=<hotplugged_memory_size>,prefault=on|off,thp=on|off”

[default: size=512M]
--memory-zone <memory-zone>...

User defined memory zone parameters

“size=<guest_memory_region_size>, file=<backing_file>, shared=on|off, hugepages=on|off, hugepage_size=<hugepage_size>, host_numa_node=<node_id>, id=<zone_identifier>, hotplug_size=<hotpluggable_memory_size>, hotplugged_size=<hotplugged_memory_size>, prefault=on|off”
--net <net>...

Network parameters

“tap=<if_name>,ip=<ip_addr>,mask=<net_mask>,mac=<mac_addr>,fd=<fd1,fd2…>,iommu=on|off,num_queues=<number_of_queues>,queue_size=<size_of_each_queue>,id=<device_id>, vhost_user=<vhost_user_enable>, socket=<vhost_user_socket_path>, vhost_mode=client|server, bw_size=,bw_one_time_burst=, bw_refill_time=, ops_size=<io_ops>, ops_one_time_burst=<io_ops>, ops_refill_time=, pci_segment=<segment_id>offload_tso=on|off, offload_ufo=on|off, offload_csum=on|off”
--numa <numa>...

Settings related to a given NUMA node

“guest_numa_id=<node_id>, cpus=<cpus_id>, distances=<list_of_distances_to_destination_nodes>, memory_zones=<list_of_memory_zones>, pci_segments=<list_of_pci_segments>”
--pci-segment <pci-segment>...

PCI Segment parameters

“pci_segment=<segment_id>, mmio32_aperture_weight=, mmio64_aperture_weight=”
--platform <platform>

num_pci_segments=<num_pci_segments>, iommu_segments=<list_of_segments>, iommu_address_width=<bits>, serial_number=<dmi_device_serial_number>, uuid=<dmi_device_uuid>, oem_strings=<list_of_strings>
--pmem <pmem>...

Persistent memory parameters

“file=<backing_file_path>,size=<persistent_memory_size>,iommu=on|off,discard_writes=on|off,id=<device_id>,pci_segment=<segment_id>”
--pvpanic

Enable pvpanic device
--rate-limit-group <rate-limit-group>...

Rate Limit Group parameters

"bw_size=<bytes>, bw_one_time_burst=<bytes>, bw_refill_time=<ms>, ops_size=<io_ops>, ops_one_time_burst=<io_ops>, ops_refill_time=<ms>, id=<device_id>"
--restore <restore>

Restore from a VM snapshot.

Restore parameters

"source_url=<source_url>, prefault=on|off, net_fds=<list_of_net_ids_with_their_associated_fds>"

source_url should be a valid URL (e.g file:///foo/bar or tcp://192.168.1.10/foo)

prefault brings memory pages in when enabled (disabled by default)

net_fds is a list of net ids with new file descriptors. Only net devices backed by FDs directly are needed as input.
--rng <rng>

Random number generator parameters “src=<entropy_source_path>,iommu=on|off”

[default: src=https://skyao.net/learning-cloud-hypervisor/dev/urandom]
--seccomp <seccomp>

[default: true] [possible values: true, false, log]
--serial <serial>

Control serial port: off|null|pty|tty|file=</path/to/a/file>|socket=</path/to/a/file>

[default: null]
--tpm <tpm>

TPM device “(UNIX Domain Socket from swtpm) socket=</path/to/a/socket>”
--user-device <user-device>...

Userspace device socket=<socket_path>,id=<device_id>,pci_segment=<segment_id>
-v...

Sets the level of debugging output
--vdpa <vdpa>...

vDPA device

“path=<device_path>,num_queues=<number_of_queues>,iommu=on|off,id=<device_id>,pci_segment=<segment_id>”
-V, --version

Print version
--vsock <vsock>

Virtio VSOCK parameters

“cid=<context_id>, socket=<socket_path>, iommu=on|off, id=<device_id>, pci_segment=<segment_id>”
--watchdog

Enable virtio-watchdog
-h, --help

Print help

ch-remote

通过 help 命令可以得到参数和参数说明:

ch-remote --help

命令	命令描述
add-device	Add VFIO device
add-disk	Add block device
add-fs	Add virtio-fs backed fs device
add-net	Add network device
add-pmem	Add persistent memory device
add-user-device	Add userspace device
add-vdpa	Add vDPA device
add-vsock	Add vsock device
boot	Boot a created VM
coredump	Create a coredump from VM
counters	Counters from the VM
create	Create VM from a JSON configuration
delete	Delete a VM
info	Info on the VM
nmi	Trigger NMI
pause	Pause the VM
ping	Ping the VMM to check for API server availability
power-button	Trigger a power button in the VM
reboot	Reboot the VM
receive-migration	Receive a VM migration
remove-device	Remove VFIO and PCI device
resize	Resize the VM
resize-zone	Resize a memory zone
restore	Restore VM from a snapshot
resume	Resume the VM
send-migration	Initiate a VM migration
shutdown	Shutdown the VM
shutdown-vmm	Shutdown the VMM
snapshot	Create a snapshot from VM
help	Print this message or the help of the given subcommand(s)

命令行选项:

选项	选项描述
`--api-socket <api-socket>`	HTTP API socket path (UNIX domain socket)
-h, –help	Print help
-V, –version	Print version