Overview

Xen 3+ supports memory “ballooning” which allows you to hot add and remove memory from a running system. It is a nice feature and has come in handy for me on many occasions.

Memory Hot Remove Example

Lets say I have a virtual machine named foo which I’ve given 1024 megs of ram. One day I notice that the system is using only 25% of this memory. I want to re-size it to 512 megs and free the rest of the memory for use by other domains but I can’t afford a downtime. No problem, Xen lets you resize memory on-the-fly.

Before we adjust the memory allocations lets see what the system looks like…

# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      256     2 r-----  49678.8
foo                                        2     1024     1 -b----   2160.2

To resize we use the ‘xm mem-set’ command. For example, to re-size our 1024M domain to 512M execute the following.

# xm mem-set foo 512
# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      256     2 r-----  49681.0
foo                                        2      512     1 -b----   2160.3

Memory Hot Add Example

We can also use the xm-mem set command to hot-add memory to a running domain, there is one catch, however. Xen can only add memory up to the amount specified by the ‘maxmem’ configuration directive in your domU config file. If no ‘maxmem’ option is specified then the maximum memory value is set to the size of the ‘memory’ option.

This is an example config file for a domain named ‘foo’ that will boot with 1024M RAM and can grow up to 4096M.

#/etc/xen/configs/foo
 
name   = "foo"
memory = 1024 
maxmem = 4096
vcpus  = 1

When we boot this domain we’ll see that its initial memory size is 1024M.

# xm create /etc/xen/configs/foo
Using config file "/etc/xen/configs/foo".
Started domain foo
 
# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      256     2 r-----  49723.1
foo                                       11     1024     1 r-----      5.1

Now we can mem-set the domain up to the value of maxmem, in our case 4096M.

xm mem-set foo 4096

# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      256     2 r-----  49725.6
foo                                       11     4096     1 -b----      8.8

Use Scenarios and Ideas

As you can see this feature provides for a great deal of flexibility in the management of your systems. I’ve found it especially helpful in dealing with live migration, you can easily shrink multiple hosts to make room for one more. You could also build a very powerful auto-scaling engine around the memory balloon and live migration features where domains can automatically re-size their memory as needed, migrating on and off of hosts according to changing demand.