Recursive Inotify Scripting With Lsyncd

Tags: , , , , , , , , , , , ,


Lsyncd is a tool which was built to keep two locations in sync with each other efficiently by only sending updates when file changes are detected with inotify events. However, lsyncd is actually quite extensible in that it that supports scripting for each of it’s various types of inotify events. This allows us to perform customized tasks when file changes are detected.

Here are a few examples:

Enforce File Permissions Recursively With Inotify

This lsyncd config will ensure that that files changed, moved or created under the defined directory have mode 777.

#/etc/lsyncd.d/chown.lsyncd
 
settings = {
	statusFile = "/tmp/chown.lsyncd.stat",
	statusIntervall = 1,
	logfacility = daemon,
}
 
chown = {
    delay = 5,
    maxProcesses = 5,
    onCreate  = "chmod 777 ^sourcePathname",
    onModify  = "chmod 777 ^sourcePathname",
    onMove    = "chmod 777 ^d.targetPathname",
    onStartup = "sysctl fs.inotify.max_user_watches=1048576; sysctl fs.inotify.max_queued_events=2097152; chmod -R 777 ^source"
}
 
sync { chown,
         source="/path/to/files",
         target="/dev/null",
}

To start lsyncd run this:

lsyncd -pidfile /var/run/chown.lsyncd /etc/lsyncd.d/chown.lsyncd

This will result in the defined “source” directory being monitored for file changes and additionally when lsyncd is started it will recursively chmod the “source” directory to ensure that any potentially missed file have the correct permissions. You may notice the sysctl commands that are being run as the “onStartup” command. This is because my watched directory is quite large, and requires adjustments to the default inotify sysctl values.

Inotify Backup – Backup Files When Changed

The below will watch your home directory for file changes and after detecting a changed file will immediately copy that file to the backup destination using rsync while appending a date-stamp to the backup file. To accomplish remote backups you could specify a remote rsync server, or use rsync+ssh with pre shared keys.

#/etc/lsyncd.d/backup.lsyncd
settings = {
	statusFile = "/tmp/backup.lsyncd.stat",
	statusIntervall = 1,
	logfacility = daemon,
}
 
backup = {
    delay = 5,
    maxProcesses = 5,
    onCreate = "rsync -a --backup --suffix=-`date +%F-%T` ^sourcePathname ^target",
    onModify = "rsync -a --backup --suffix=-`date +%F-%T` ^sourcePathname ^target",
}
 
sync { backup,
       source="/home",
       target="/var/backups/"
}

To start lsyncd run this:

lsyncd -pidfile /var/run/backup.lsyncd /etc/lsyncd.d/backup.lsyncd

I chose lsyncd over the various alternatives out there because to me it made the most sense. I liked that it is built to run as a daemon, and does the vast majority of the heavy lifting for me. Watching a directory recursively was a must have for me, and it requires a minimal amount of scripting for most uses.

5 Responses to “Recursive Inotify Scripting With Lsyncd”

  1. Inheriting file permission Says:

    […] which addresses this… http://techblog.shanock.com/articles…of-a-directory And another; http://backdrift.org/recursive-inoti…ng-with-lsyncd And a Q&A from another site; http://superuser.com/questions/15191…rent-directory The short […]

  2. Suresh Meeraj Says:

    I am using fedora 18
    I am not finding the this “#/etc/lsyncd.d/chown.lsyncd” on my machine

    please help me out

    [Reply]

    Kris Griebe Reply:

    chown.lsyncd is something custom he created, it would not be on your system by default.

    I would guess the lsyncd package isn’t a standard package on Fedora 18 either, so you’d have to install it via yum:

    https://code.google.com/p/lsyncd/

    [Reply]

  3. Hermeticus Says:

    i had trouble to get lsyncd vars (^source…) replaced properly.
    After having a look at the source, it seems that while “quotation marks” are accepted in the config file no translation of the variables will happen.
    Buf if you use ‘single quotation’ marks or [[double square brackets]] as delimters then the translation will happen.

    [Reply]

  4. Jose Says:

    My Notes broke when I upgraded to 64-bit Fedora 18. I have tried both 8.5.3 and 9 beta. I see two prelobms. In /tmp/_lap5185859786106604928pid.err I see error: list of process IDs must follow -pUsage: ps [options] Try ps help or ps help for additional help text.For more details see ps(1).Which seems to come from /tmp//tmp/_lap1446502474866698423wrapper#!/bin/shexport LANG=en_US.UTF-8uxterm -T -e echo;/tmp/_lap2210317580084186344invoker sleep 1pid=`ps -ef | egrep /tmp/_lap2210317580084186344invoker | grep /bin/sh | awk {print $2}’`sleep 1rc=0while [ $rc -eq 0 ]do sleep 1 ps -p $pid >/tmp/_lap2867327354443588968pid.out 2>/tmp/_lap5185859786106604928pid.err rc=$?doneIs it possible to modify the file that generates these /tmp startup scripts? Also, when using 9 I get a text license file that I try to accept, but I see the following in the ~/lotus/notes/data/workspace/logs/error-log-0.xml:Any ideas?

    [Reply]

Join the Conversation