qemu-kvm-command-options

Posted on Edited on In

Standard Options

-boot [order=drives][,once=drives][,menu=on|off][,splash=sp_name][,splash-time=sp_time][,reboot-timeout=rb_timeout][,strict=on|off]

Specify boot order drives as a string of drive letters. Valid drive letters depend on the target architecture. The x86 PC uses: a, b (floppy 1 and 2), c (first hard disk), d (first CD-ROM), n-p (Etherboot from network adapter 1-4), hard disk boot is the default. To apply a particular boot order only on the first startup, specify it via once. Note that the order or once parameter should not be used together with the bootindex property of devices, since the firmware implementations normally do not support both at the same time.

-machine [type=]name[,prop=value[,…]]

Select the emulated machine(chipset) by name. Use -machine help to list available machines.For architectures which aim to support live migration compatibility across releases, each release will introduce a new versioned machine type. For example, the 2.8.0 release introduced machine types “pc-i440fx-2.8” and “pc-q35-2.8” for the x86_64/i686 architectures. -machine pc-i440fx-rhel7.6.0,accel=kvm,usb=off,dump-guest-core=off

-cpu model

Select CPU model (-cpu help for list and additional feature selection), -cpu help to check availabe cpu and flags, enable or disable or use default.
-cpu IvyBridge,pcid=on,movbe=on,hypervisor=on,arat=on,tsc_adjust=on,avx2=on,rdseed=on,clflushopt=on,abm=on,3dnowprefetch=on,f16c=off,smep=off,erms=off,xsaveopt=off without -cpu default is used that’s model name : QEMU Virtual CPU version 2.5+

-daemonize

Daemonize the QEMU process after initialization. QEMU will not detach from standard IO until it is ready to receive connections on any of its devices. This option is a useful way for external programs to launch QEMU without having to cope with initialization race conditions.

-no-shutdown

Don’t exit QEMU(qemu process is still there) on guest shutdown, but instead only stop the emulation. This allows for instance switching to monitor to commit changes to the disk image. but this is not working for libvirt as it will kill qemu process when shutdown from guest finished.

-nodefaults

Don’t create default devices. Normally, QEMU sets the default devices like serial port, parallel port, virtual console, monitor device, VGA adapter, floppy and CD-ROM drive and others. The -nodefaults option will disable all those default devices if -nodefaults is used, you must set vga adapter explicitly(-vga std), otherwise VNC is not working!!!

-S

Do not start CPU at startup (you must type ‘c’ in the monitor), this is mostly used for migration, at dst host, start qemu with such option, then we can copy memory from source to dst

-object typename[,prop1=value1,…]

Create a new object of type typename setting properties in the order they are specified. Note that the ‘id’ property must be set. These objects are placed in the ‘/objects’ path

-no-user-config

The -no-user-config option makes QEMU not load any of the user-provided config files on sysconfdir(/etc/qemu/qemu.conf)

-global driver.prop=value

In particular, you can use this to set driver properties for devices which are created automatically by the machine model, example -global kvm-pit.lost_tick_policy=delay

1
2
3
4
# get all properties of a given driver 
/usr/libexec/qemu-kvm -device kvm-pit,help
kvm-pit.iobase=uint32
kvm-pit.lost_tick_policy=LostTickPolicy

-rtc [base=utc|localtime|datetime][,clock=host|rt|vm][,driftfix=none|slew]

Specify base as utc or localtime to let the RTC start at the current UTC or local time, respectively. localtime is required for correct date in MS-DOS or Windows.

By default the RTC is driven by the host system time. This allows using of the RTC as accurate reference clock inside the guest, specifically if the host time is smoothly following an accurate external reference clock, e.g. via NTP. If you want to isolate the guest time from the host, you can set clock to rt instead, which provides a host monotonic clock if host support it. timezones abbre and rtc, pit, hpet

check guest clock source by cat /sys/devices/system/clocksource/clocksource0/current_clock

-msg [timestamp[=on|off]][,guest-name[=on|off]]

Control error message format of qemu process

memory

For each memory(main ram, device ram), Qemu will create memory backend object automatically, QEMU has alternative memory backends.

  • memory-backend-ram – The default backend
  • memory-backend-file – Memory backed by a file, typcially used for non volatile memories (to save the contents when powering off the machine)
  • memory-backend-memfd – Anonymous memory file backend

-m [size=]megs[,slots=n,maxmem=size]

Sets guest startup RAM(main ram) size to megs megabytes. Optionally, a suffix of “M” or “G” can be used to signify a value in megabytes or gigabytes respectively. Optional pair slots, maxmem could be used to set amount of hotpluggable memory slots and maximum amount of memory. Note that maxmem must be aligned to the page size.

For example, the following command-line sets the guest startup RAM size to 1GB, creates 3 slots to hotplug additional memory and sets the maximum memory the guest can reach to 4GB:qemu-system-x86_64 -m 1G,slots=3,maxmem=4G

If slots and maxmem are not specified, memory hotplug won’t be enabled and the guest startup RAM will never increase.

By default, Qemu will create a memory backend(memory-backend-ram) object(named pc.ram) for the main ram automatically if user does not set memory backend object(pc.ram) explicitly.

-object memory-backend-ram,id=id,merge=on|off,dump=on|off,share=on|off,prealloc=on|off,size=size,host-nodes=host-nodes,policy=default|preferred|bind|interleave

Creates a memory backend object, which can be used to back the guest RAM. Memory backend objects offer more control than the -m option that is traditionally used to define guest RAM, with host-nodes can specify where guest ram comes from(which numa node).
bind: Strict policy means that the allocation will fail if the memory cannot be allocated on the target node.
interleave: Memory pages are allocated across nodes specified by a nodeset, but are allocated in a round-robin fashion.
preferred: Memory is allocated from a single preferred memory node. If sufficient memory is not available, memory can be allocated from other nodes.

-object memory-backend-file,id=id,size=size,mem-path=dir,share=on|off,discard-data=on|off,merge=on|off,dump=on|off,prealloc=on|off,host-nodes=host-nodes,policy=default|preferred|bind|interleave,align=align,readonly=on|off

Creates a memory file backend object(mem-path is required), which can be used to back the guest RAM with huge pages.

The id parameter is a unique ID that will be used to reference this memory region in other parameters, e.g. -numa, -device nvdimm, etc.

The size option provides the size of the memory region, and accepts common suffixes, e.g. 500M.

The mem-path provides the path to either a shared memory or huge page filesystem mount.

No Explicit memory backend

1
2
# by default, memory is not prealloc, memory is grabed from host only when it's accessed.
-m 512M

Explicit memory backend ram more control

1
2
3
4
5
6
7
8
9
10
# new qemu version, memory-backend as the key
# guest ram bind to host numa node0, if no sufficient memory, it fails!!!
-object memory-backend-ram,id=pc.ram,size=512M,dump=on,share=on,prealloc=on,host-nodes=0,policy=bind \
-machine memory-backend=pc.ram \
-m 512M

# old qemu version, memdev as the key
-object memory-backend-ram,id=pc.ram,size=512M,dump=on,share=on,prealloc=on \
-numa node,nodeid=0,memdev=pc.ram \ 
-m 512M

Explicit memory backend file to use hugepage

Qemu will create this file automatically, but not delete it when qemu exits

1
2
3
4
5
6
7
8
9
# new qemu version
-object memory-backend-file,id=pc.ram,size=512M,mem-path=/dev/hugepages/libvirt/qemu/centos,prealloc=on,share=on,host-nodes=0,policy=bind \
-machine memory-backend=pc.ram \
-m 512M

# old qemu version
-object memory-backend-file,id=pc.ram,size=512M,mem-path=/dev/hugepages/libvirt/qemu/centos,prealloc=on,share=on,host-nodes=0,policy=bind \
-numa node,nodeid=0,memdev=pc.ram \ 
-m 512M

CPUS and Numa

-smp [[cpus=]n][,maxcpus=maxcpus][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]

Simulate a SMP system with ‘n’ CPUs initially present on the machine type board. On boards supporting CPU hotplug, the optional ‘maxcpus’ parameter can be set to enable further CPUs to be added at runtime. When both parameters are omitted, the maximum number of CPUs will be calculated from the provided topology members and the initial CPU count will match the maximum number. When only one of them is given then the omitted one will be set to its counterpart’s value.

-smp 8,sockets=2,cores=2,threads=2,maxcpus=8

Either the initial CPU count, or at least one of the topology parameters must be specified. Values for any omitted parameters will be computed from those which are given.

-numa node[,mem=size][,cpus=firstcpu[-lastcpu]][,nodeid=node][,initiator=initiator]
-numa node[,memdev=id][,cpus=firstcpu[-lastcpu]][,nodeid=node][,initiator=initiator]
-numa dist,src=source,dst=destination,val=distance
-numa cpu,node-id=node[,socket-id=x][,core-id=y][,thread-id=z]
-numa hmat-lb,initiator=node,target=node,hierarchy=hierarchy,data-type=tpye[,latency=lat][,bandwidth=bw]
-numa hmat-cache,node-id=node,size=size,level=level[,associativity=str][,policy=str][,line=size]

Define a NUMA node and assign RAM and VCPUs to it. Set the NUMA distance from a source node to a destination node. Set the ACPI Heterogeneous Memory Attributes for the given nodes.

Legacy CPU assigment
Legacy VCPU assignment uses ‘cpus’ option where firstcpu and lastcpu are CPU indexes. Each ‘cpus’ option represent a contiguous range of CPU indexes (or a single VCPU if lastcpu is omitted). A non-contiguous set of VCPUs can be represented by providing multiple ‘cpus’ options. If ‘cpus’ is omitted on all nodes, VCPUs are automatically split between them.

New CPU assignment Way
‘cpu’ option is a new alternative to ‘cpus’ option which uses ‘socket-id|core-id|thread-id’ properties to assign CPU objects to a node using topology layout properties of CPU.

memory assignment way(legacy and new)
Legacy ‘mem’ assigns a given RAM amount to a node (not supported for 5.1 and newer machine types). ‘memdev’ assigns RAM from a given memory backend device to a node. If ‘mem’ and ‘memdev’ are omitted in all nodes, RAM is split equally between them.

New Way for cpu(cpu) and memory(mdev)

1
2
3
4
5
6
7
8
9
-M pc \
-m 2G,slots=2,maxmem=4G \
-object memory-backend-ram,size=1G,id=m0 \
-object memory-backend-ram,size=1G,id=m1 \
-numa node,nodeid=0,memdev=m0 \
-numa node,nodeid=1,memdev=m1 \
-smp 2,sockets=2,maxcpus=2  \
-numa cpu,node-id=0,socket-id=0 \
-numa cpu,node-id=1,socket-id=1

Legency CPU(cpus) and New memory way(mdev)

1
2
3
4
5
6
7
-M pc \
-m 2G,slots=2,maxmem=4G \
-object memory-backend-ram,size=1G,id=m0 \
-object memory-backend-ram,size=1G,id=m1 \
-smp 2,sockets=2,maxcpus=2  \
-numa node,nodeid=0,memdev=m0,cpus=0 \
-numa node,nodeid=1,memdev=m1,cpus=1

Legency CPU(cpus) and legency memory way(mem)

1
2
3
4
5
-M pc \
-m 2G,slots=2,maxmem=4G \
-smp 2,sockets=2,maxcpus=2  \
-numa node,nodeid=0,mem=1G,cpus=0 \
-numa node,nodeid=1,mem=1G,cpus=1

Note that the -numa option doesn’t allocate any of the specified resources, it just assigns existing resources to NUMA nodes. This means that one still has to use the -m, -smp options to allocate RAM and VCPUs respectively.

-sandbox arg[,obsolete=string][,elevateprivileges=string][,spawn=string][,resourcecontrol=string]

Enable Seccomp mode 2 system call filter. ‘on’ will enable syscall filtering and ‘off’ will disable it. The default is ‘off’. It disables system calls that are not needed by QEMU, thereby reducing the host kernel attack surface.

obsolete=string

Enable Obsolete system calls

elevateprivileges=string

Disable set*uid|gid system calls

spawn=string

Disable *fork and execve

resourcecontrol=string

Disable process affinity and schedular priority

NOTE

  • qemu does not proivde parameter to bind host’s physical cpu to vcpu, vcpu affinity is set outside of qemu either by cgroup as we create cpu cgroup for each vm or cpuset_setaffinity() if cgroup is not available

migration

-incoming tcp:[host]:port[,to=maxport][,ipv4=on|off][,ipv6=on|off]
-incoming rdma:host:port[,ipv4=on|off][,ipv6=on|off]

Prepare for incoming migration, listen on a given tcp port.

-incoming unix:socketpath

Prepare for incoming migration, listen on a given unix socket.

-incoming fd:fd

Accept incoming migration from a given filedescriptor.

1
2
3
4
5
6
7
8
9
10
11
12
13
# -S -incoming tcp:0.0.0.0:10000 is used for migration at dst host
# start qemu at dst host, now cpu is stopped, listn on 10000 for migration
$/usr/libexec/qemu-kvm $args -S -incoming tcp:0.0.0.0:10000

qmp_shell/> query-status
{
    "running": false,
    "singlestep": false,
    "status": "inmigrate"
}

# then at src host, you can start migration process
# after migration completed, start vm at dst host

Device Emulation

usb options

-usb

Enable USB emulation on machine types with an on-board USB host controller (if not enabled by default). Note that on-board USB host controllers may not support USB 3.0. In this case -device qemu-xhci can be used instead on machines with PCI.

-usbdevice devname

Add the USB device devname, and enable an on-board USB controller if possible and necessary (just like it can be done via -machine usb=on). Note that this option is mainly intended for the user’s convenience only. More fine-grained control can be achieved by selecting a USB host controller (if necessary) and the desired USB device via the -device option instead. For example, instead of using -usbdevice mouse it is possible to use -device qemu-xhci -device usb-mouse to connect the USB mouse to a USB 3.0 controller instead (at least on machines that support PCI and do not have an USB controller enabled by default yet).

1
2
3
4
5
6
7
8
# easy way
# use on-board usb controller
-usb \
-usbdevice tablet \

# more fine-grained control way(use usb3.0)
-device qemu-xhci \
-device usb-tablet \

-device usb-tablet

Pointer device that uses absolute coordinates (like a touchscreen). This means QEMU is able to report the mouse position without having to grab the mouse. Also overrides the PS/2 mouse emulation when activated. with this enabled, mouse cursor follows mous position right way.

block options

-blockdev option[,option[,option[,…]]]
Define a new block driver node(backend) only. Some of the options apply to all block drivers, other options are only accepted for a specific block driver.

Options that expect a reference to another node (e.g. file) can be given in two ways. Either you specify the node name of an already existing node (file=node-name), or you define a new node inline, adding options for the referenced node after a dot (file.filename=path,file.aio=native).

A block driver node created with -blockdev can be used for a guest device by specifying its node name for the drive property in a -device argument that defines a block device.

Valid options for any block driver node:

  • driver

    Specifies the block driver to use for the given node.

  • node-name

    This defines the name of the block driver node by which it will be referenced later. The name must be unique, i.e. it must not match the name of a different block driver node, or (if you use -drive as well) the ID of a drive.

Driver-specific options for file

  • filename

    The path to the image file in the local filesystem -blockdev driver=file,node-name=disk,filename=disk.img

Driver-specific options for raw

  • file

    Reference to or definition of the data source block driver node (e.g. a file driver node), -blockdev driver=raw,node-name=disk,file=disk_file

Driver-specific options for qcow2

  • file

    Reference to or definition of the data source block driver node (e.g. a file driver node) -blockdev driver=qcow2,node-name=hda,file=my_file,overlap-check=none,cache-size=16777216

-drive option[,option[,option[,…]]]

Define a new drive. This includes creating a block driver node (the backend) as well as a guest device, and is mostly a shortcut for defining the corresponding -blockdev and -device options.

-drive accepts all options that are accepted by -blockdev. In addition, it knows the following options:

  • file=file

    This option defines which disk image to use with this drive

  • if=interface

    This option defines on which type on interface the drive is connected. Available types are: ide, scsi, sd, mtd, floppy, pflash, virtio, none

  • bus=bus,unit=unit

    These options define where is connected the drive by defining the bus number and the unit id.

  • index=index

    This option defines where the drive is connected by using an index in the list of available connectors of a given interface type, cdrom and disk share the index!!!

  • media=media

    This option defines the type of the media: disk or cdrom

  • format=format

    Specify which disk format will be used rather than detecting the format. Can be used to specify format=raw to avoid interpreting an untrusted format header.

  • iops=i,iops_rd=r,iops_wr=w

    Specify request rate limits in requests per second, either for all request types or for reads or writes only.

1
2
3
4
5
6
7
8
9
10
11
12
# cdrom
qemu-system-x86_64 -drive file=file,index=2,media=cdrom

# fda, fdb
qemu-system-x86_64 -drive file=file,index=0,if=floppy
qemu-system-x86_64 -drive file=file,index=1,if=floppy

# index0 hda, index1 hdb as if=ide by default.
qemu-system-x86_64 -drive file=file,index=0,media=disk
qemu-system-x86_64 -drive file=file,index=1,media=disk
qemu-system-x86_64 -drive file=file,index=2,media=disk
qemu-system-x86_64 -drive file=file,index=3,media=disk

FAQ

what’s chipset?

A chipset is a set of electronic components in an integrated circuit known as a "Data Flow Management System" that manages the data flow between the processor, memory and peripherals. It is usually found on the motherboard. Chipsets are usually designed to work with a specific family of microprocessors. Because it controls communications between the processor and external devices, the chipset plays a crucial role in determining system performance.

add console to qemu

With this, we can access console from pty directly

1
2
3
4
5
6
7
8
9
10
11
12
13
# run qemu-kvm with daemon, then later connect to its console
-D /tmp/qemu_vm.log \
-daemonize \
-device virtio-serial-pci,id=virtio-serial0 \
-chardev pty,id=charconsole1 -device virtconsole,chardev=charconsole1,id=console1 \

$ cat /tmp/qemu_vm.log
qemu-kvm: -chardev pty,id=charconsole1: char device redirected to /dev/pts/1 (label charconsole1)

$ screen /dev/pts/1

# quit from console(terminate screen session)
ctrl + a, then press \

REF