Version Controlled etc
From TLUGWiki
Contents |
[edit] Introduction
In this howto I will try to explain how to create and manage a version controlled /etc. This can be done with almost any version control system. For the purposes of this tutorial I will use Subversion, I reckon rcs may also be a good choice (dispatch-conf in Gentoo uses that).
There is currently two methods to achieve this. I will document my initial method first, but this has some problems which are avoided by the second method. Note that the problems don't go away, they are merely avoided!
[edit] Argument for Version Controlled /etc
It has happened to everyone doing system administration. You have a perfectly working config file. Some user requests additional functionality, you make the reguired change in the config file and it blows up in your face, and for the life of you you can't figure out what you have changed.
Or you simply updated some services and the update clobbered your config file. This happens all too often if you are a careless Gentoo user, or is in a hurry. Or perhaps you are using some other binary distro that just clobbers config files out of principle (someone forgot to add a few config protection indicators in some rpm package).
[edit] dispatch-conf (Gentoo)
Why do I explicitly mention dispatch-conf? Simply because it's provided by Gentoo as a method for tracking changes of config files. It might be a better solution for Gentoo users, but at this point in time I'm not sure how well it handles edits of files between runs of dispatch-conf (usually run directly after major updates when emerge indicates that there are config files to update). It also makes use of either a diff-based system to track changes, or it uses rcs.
[edit] Advantages of a version controlled /etc
- You can easily track changes that you've made.
- You can roll-back to previous configs.
- If some automated process clobbers your changes you can easily fix it.
- Accidental changes can be reverted very quickly.
- Will hopefully make you think about every change you make.
[edit] Disadvantages of a version controlled /etc
- It takes some time to set up and configure.
- You need to remember to add new config files.
- You need to remember to commit changes.
- Extra space required for version information
- /etc will take about double the space from normal
- The repository will also consume some space (My initial repository is about 3M compared to a 9M /etc)
- rc-update -s complains about the .svn directories and all files inside of that, it still functions correctly though.
[edit] Initial (dangerous) method
This method has the disadvantage that you need to manually fix all the ownerships and permissions after the initial checkout.
[edit] Overview
In order to store config files in a version-controlled manner one need a few things:
- A version control system (Subversion in this howto).
- A list of system-changed files that should not be placed under version control (like /etc/mtab).
- Some patience.
The basic process is as follows:
- Create a repository.
- Import your existing /etc directory into the repository.
- Check out the repository to somewhere (not /etc, something like /tmp/etc).
- Swiwel the checked-out version and your real /etc.
- Manage your config files, remembering to commit working changes to your config files.
All in all it's a bit more effort to maintain your config files but the advantage of being able to go back and check what your config was at any point in the past (well, back to when you configured a version controlled etc) outweighs this extra effort for me.
[edit] Creating the repository
Considering that we need to be root in order to edit config files I will maintain the repository as root. This may or may not be a good idea. Since only root will ever commit into this repository placing it somewhere where only root can get to is probably a good idea. I like /root for that.
# svnadmin create /root/.version_controlled_etc
Considering that perhaps there are other locations too that might need to be version controlled (/var/qmail/control, /opt/MailScanner/etc) and so forth, it's a good idea to also create a special "project folder" for /etc:
# svn mkdir file:///root/.version_controlled_etc/etc -m "Create location for storing /etc config files" Committed revision 1. #
[edit] Importing your existing /etc
At this point in time I suggest making a backup archive of your entire /etc (just in case):
# tar czf /root/etc-backup-$(date +%Y%m%d).tgz /etc #
You are now ready to perform the import:
# svn import . file:///root/.version_controlled_etc/etc/ -m "Import existing /etc" ... Adding services Adding rpc Adding mail.rc Adding xpdfrc ... Adding X11/app-defaults/Xgc Adding X11/app-defaults/Xditview-chrtr Adding X11/app-defaults/Viewres Adding X11/app-defaults/XClipboard ... Adding asciidoc/xhtml-deprecated.conf Committed revision 2. #
After a while your entire /etc is now committed into the repository. You will note that I very conveniently ignored the standard {trunk,tags,branches} structure that is common to subversion repositories. My argument for that is simply that we are not going to make point releases of our etc (unless you are suicidal of course).
[edit] Checking out the committed repository
This is done as normal, we will check out to /tmp_etc. This is a good location since
# svn co file:///root/.version_controlled_etc/etc/ /tmp_etc ... A /tmp_etc/bash A /tmp_etc/bash/bashrc A /tmp_etc/bash/bash_logout A /tmp_etc/bonobo-activation ... A /tmp_etc/asciidoc/xhtml11-quirks.conf Checked out revision 2. #
At this point in time we now have a serious problem though:
# ls -l /tmp_etc/shadow -rw-r--r-- 1 root root 758 Dec 14 13:45 /tmp_etc/shadow # ls -l /etc/shadow -rw------- 1 root root 758 Dec 12 15:29 /etc/shadow #
At this point in time you need to manually go through all files and directories in /tmp_etc and fix their permissions and ownerships. This isn't a nice process. I've skipped this and never even performed the swiwel described lower down. You want to use a combination of find, stat, chmod and chown. awk and sed will probably also come in handy. If you manage to script this, please paste the script in here.
[edit] Removing unneeded files from version control
Some files, like mtab does not need to be under version control. It's not a serious problem if they are, but they will very likely cause a large number of unneeded commits and generally irritate you. To remove a file from version control, do the following (for mtab):
# pwd
/tmp_etc
# svn status -v mtab
2 2 root mtab
# cp mtab tmp_mtab
# svn del mtab
# mv tmp_mtab mtab
# svn propedit svn:ignore
-- add mtab on it's own line (svn:ignore is a list of glob expressions)
# svn commit -N mtab . -m "Remove mtab from version control"
Sending .
Deleting mtab
Committed revision 3.
#
mtab is now no longer under version control. Please note that the svn del command actually deleted the file from your working copy! For this reason you should make a backup copy first and then restore it afterwards. The propedit is there to prevent svn status from showing mtab as an unknown file. You can add more filters (to ignore the Gentoo config update files for example).
[edit] Swiweling /tmp_etc and /etc
This is a rather simple operation.
There is some danger unfortunately. What happens if a config file changed from when you started the whole process untill now? For this reason it might be worthwhile to perform this entire switch from a livecd where you can be sure nothing in etc will change. I personally think going to those extremes is overkill.
# mv /etc /old_etc && mv /tmp_etc /etc #
It should be almost instantanious. You can now happily use svn commands like you would inside a normal svn working copy to manipulate your config files.
[edit] Alternative (safer?) method
There seems to be a safer method that avoids moving things around. Create your repository as above and then follow the instructions here. Note that we merely avoid the permissions problems here. There is also the advantage that we don't actually switch the currently in use /etc with another one, potentially avoiding some relatively dangerous and unexpected situations.
[edit] Create a etc directory within the repository.
# svn mkdir file:///root/.version_controlled_etc/etc -m "Create location for storing /etc config files" Committed revision 1. #
This is still the same as above. The difference is now that we check this out directly into /etc, like so:
[edit] Check out the repository
# cd /etc # svn co file:///root/.version_controlled_etc/etc . Checked out revision 1. #
At this point in time executing "svn status" will show you all the files directly in /etc along with question marks (unknown status).
[edit] Adding files into the repository
We are now ready to add the files that we do want in version control. So let's start with everything in init.d, some local files and some other relatively important stuff:
# cd /etc # svn add -N init.d init.d/* X11 X11/xorg.conf fstab A init.d A init.d/gpm A init.d/nfs A init.d/xdm A init.d/xfs A init.d/ntp-client ... A init.d/alsasound A X11 A X11/xorg.conf A fstab #
Note the use of -N to prevent svn from recursively adding directories. If you wanted to recursively add init.d but not X11 you would have used two seperate svn commands.
Now the files merely needs to be committed into the repository:
# svn commit -m "Import init.d*, X11/, X11/xorg.conf and fstab into the repository" Adding X11 Adding X11/xorg.conf Adding fstab Adding init.d Adding init.d/acpid Adding init.d/alsasound ... Adding init.d/xdm Adding init.d/xfs Transmitting file data ..................................................................................... Committed revision 2. #
[edit] Maintaining /etc within version control
[edit] Keep the repository up to date
You need to be aware that you will need to keep your repository up to date. Random changes should be committed when you are done and you should put accurate log entries. Entries like "Changed from foo to bob" won't help you. Use entries like "Switched algorithmic scheduling in random package from foo to bob" is much more descriptive.
[edit] Changing files
If you find that a file keeps on changing from somewhere, then it might be that that file doesn't belong in the version control (perhaps it's auto-generated or something). In this case, follow the procedures outlined above to remove the file from version control.
[edit] cron to the rescue
Forgetting to commit config files can lead to nasty problems. It's possibly a good idea to add a command to cron to check periodically that all files in /etc is in subversion, and that all changes is committed. An entry like the following should do the job nicely (permitting cron and your mail system has been correctly configured to be able to send you mail):
0 0 * * * /usr/bin/svn status /etc
Or if you are feeling brave:
0 0 * * * svn status /etc | awk '$1=="!" { system("svn del " $NF); } $1=="?" { if (!$NF ~ "^\._cfg") system("svn add " $NF); } END { system("svn commit -m autocommit /etc")l }'
Note that the if (! $NF ~ "^\._cfg") part is Gentoo specific and just prevents one from accidentally adding those ._cfg* files into svn ... which causes problems for commands such as etc-update :(.
If you are using this on a desktop or a notebook that isn't always switched on you should probably place the above inside a script and place it inside /etc/cron.daily and make it executable. This will allow cron to run it at any time of the day permitting at least 24 hours has passed since it was last run. So if you boot up and the script should have run 3 hours ago it will run just after bootup instead.
[edit] Permissions, Ownerships and backups
It should be noted that this repository will not suffice as a bacup method. Yes, it will ease recovering config files but due to the fact that no permission or ownerships are stored along with the files (well, I suppose it's possible to add them as properties and then script a restore - and most probably an add as well).
Thus this version control method will only protect you from _content_ changes and won't quarantee any of the "security" stuff.
If you add permissions/ownerships as file properties into svn it's probably also a good idea to create a script to check that the actual permissions/ownerships of /etc match up with those in the repository. Any takers?
Just as an example:
#!/bin/bash
#
# Script for saving and restoring *nix file attributes (mode and owner) to the Subversion
# properties.
usage() {
echo "Usage: $0 [-s] [-r] <file> ..."
}
set -e
TEMP=`getopt -o sr --long save,restore -n '$0' -- "$@"`
if [ $? != 0 ] ; then usage ; exit 1 ; fi
eval set -- "$TEMP"
DO_SAVE="true"
while true ; do
case "$1" in
-s|--save) DO_SAVE="true" ; shift ;;
-r|--restore) DO_SAVE="false" ; shift ;;
--) shift ; break ;;
esac
done
if [ "$1" == "" ]
then
usage
exit 1
fi
if [ "$DO_SAVE" != "true" ]
then
# Restore attributes
while [ "$1" != "" ]
do
FILE=$1
shift
if [ -e "$FILE" ]
then
SPECIAL=`svn propget svn:special "$FILE"`
if [ "$SPECIAL" == "" ]
then
OWNER=`svn propget lin:owner "$FILE"`
GROUP=`svn propget lin:group "$FILE"`
MODE=`svn propget lin:mode "$FILE"`
chown "$OWNER:$GROUP" "$FILE"
chmod "$MODE" "$FILE"
fi
fi
done
else
# Save attributes
while [ "$1" != "" ]
do
FILE=$1
shift
if [ -e "$FILE" ]
then
TYPE=`stat -c %F "$FILE"`
if [ "$TYPE" != "symbolic link" ]
then
OWNER=`stat -c %U "$FILE"`
GROUP=`stat -c %G "$FILE"`
MODE=`stat -c %a "$FILE"`
svn propset lin:owner "$OWNER" "$FILE"
svn propset lin:group "$GROUP" "$FILE"
svn propset lin:mode "$MODE" "$FILE"
fi
fi
done
fi
