How to Hot Add/Remove Memory from a Xen Domain
Xen 3+ supports memory “ballooning” which allows you to hot add and remove memory from a running system. It is a nice feature and has come in handy for me on many occasions.
Memory Hot Remove Example
Lets say I have a virtual machine named foo which I’ve given 1024 megs of ram. One day I notice that the system is using only 25% of this memory. I want to re-size it to 512 megs and free the rest of the memory for use by other domains but I can’t afford a downtime. No problem, Xen lets you resize memory on-the-fly.
Before we adjust the memory allocations lets see what the system looks like…
# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 256 2 r----- 49678.8 foo 2 1024 1 -b---- 2160.2
To resize we use the ‘xm mem-set’ command. For example, to re-size our 1024M domain to 512M execute the following.
# xm mem-set foo 512 # xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 256 2 r----- 49681.0 foo 2 512 1 -b---- 2160.3
Memory Hot Add Example
We can also use the xm-mem set command to hot-add memory to a running domain, there is one catch, however. Xen can only add memory up to the amount specified by the ‘maxmem’ configuration directive in your domU config file. If no ‘maxmem’ option is specified then the maximum memory value is set to the size of the ‘memory’ option.
This is an example config file for a domain named ‘foo’ that will boot with 1024M RAM and can grow up to 4096M.
#/etc/xen/configs/foo name = "foo" memory = 1024 maxmem = 4096 vcpus = 1
When we boot this domain we’ll see that its initial memory size is 1024M.
# xm create /etc/xen/configs/foo Using config file "/etc/xen/configs/foo". Started domain foo # xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 256 2 r----- 49723.1 foo 11 1024 1 r----- 5.1
Now we can mem-set the domain up to the value of maxmem, in our case 4096M.
xm mem-set foo 4096
# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 256 2 r----- 49725.6 foo 11 4096 1 -b---- 8.8
Use Scenarios and Ideas
As you can see this feature provides for a great deal of flexibility in the management of your systems. I’ve found it especially helpful in dealing with live migration, you can easily shrink multiple hosts to make room for one more. You could also build a very powerful auto-scaling engine around the memory balloon and live migration features where domains can automatically re-size their memory as needed, migrating on and off of hosts according to changing demand.