native zfs for linux on proxmox and zfs eating memory

original url: http://ispire.me/native-zfs-for-linux-on-proxmox/

 

First of all, get you system up to date:

apt-get update && apt-get upgrade -y

Install following utilities/dependencies to your System:

apt-get -y install autoconf libtool git
apt-get -y install build-essential gawk alien fakeroot zlib1g-dev uuid uuid-dev libssl-dev parted pve-headers-$(uname -r)

grab the latest source directory of SPL compile and install the compiled .deb packages:

cd /opt
git clone https://github.com/zfsonlinux/spl.git
cd spl
./autogen.sh
./configure
make deb
dpkg -i *.deb

check if spl loads correctly by entering:

modprobe spl

now do the same with ZFS source:

cd /opt
git clone https://github.com/zfsonlinux/zfs.git
cd zfs
./autogen.sh
./configure
make deb
dpkg -i *.deb

check if zfs is working:

modprobe zfs

now add the init script to your system:

update-rc.d zfs defaults

and reboot your system:

reboot

After reboot we are ready to build our Storage Pool:

zpool create -f -o ashift=12 storage mirror /dev/sdb /dev/sda

And add some tunings:

zfs set compression=on storage
zfs set sync=disabled storage
zfs set primarycache=all storage
zfs set atime=off storage
zfs set checksum=off storage

Important Caution note:
Deduplication feature requires up to 5 GB RAM per Terrabyte Storage Space, so if you cannot afford this amount of exclusive RAM disable dedup by entering:

zfs set dedup=off storage

List your Pool created before:

zpool list storage

NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
storage 1.81T 20.5G 1.79T 1% 1.00x ONLINE –

If you see something like this above we are Done!
Finally add your newly created Pool /storage to you Proxmox GUI. When adding new VM’s don’t forget to use RAW disk images and write back for your virtual disks as cache feature to get the huge speedy advantage of your zfs pool.

 

 

— ZFS EATING MEMORY

Todays solution is: If ZFS is eating your memory. In a production usage of ZFS especially when using my native ZFS on Linux article one sympthom that could occur is you ran out of of memory in short time.

Since ZFS is originally designed to run stand alone on a server,
using its total memory (default arc max value is 80% of total mem) and we are need most of the memory for our KVM instances, we have to cap the memory usage a little bit.

So if you realize high I/O writes which is eating your memory,
you can solve this by capping the arc memory limit to a lower value on your host machine.

That is what we are going to do now!

Set zfs_arc_min and zfs_arc_max memory limit

Create a file in /etc/modprobe.d/zfs.conf and add and adjust the zfs_arc_min and zfs_arc_max mem parameters:

# Min 2048MB / Max 4096 MB Limit
options zfs zfs_arc_min=2147483648
options zfs zfs_arc_max=4294967296

Reboot your machine at that should do it.