French ↓ Menu ↓

# custom website synchronisation with mobileme (2)

I already talked about how I synchronized my website with mobileme. I ameliorated this script in order to make it incremental.

Here is my new script, it first create a map which associate to each file its hash. After that it compare this file to the remote one. Then for each different file, update the content.

Even with this script I also have some problem. Mostly due to ‘webdav’ issues. For example, renaming a folder work really badly (on Linux at least). I use webdavfs. For example:

`mv folder folder2`

It returns OK and I’ve got:

`$ ls folder folder2`

Booh….

In order to handle most webdav issues I use a framework in zsh. It handle almost all except the correct renaming of folder. Working on it… Anyway here is the code I use.

`#!/usr/bin/env zsh`

function samelineprint { print -n -P – "\r$*" }

# avec 1 essai par seconde: 300 = 5 minutes

maxessais=300

# try to create a directory until success

function trymkdir { target=“$1” print – mkdir -p $target local essai=1 while ! mkdir -p $target; do samelineprint "Echec: essai n°$essai" ((essai++)) ((essai>maxessais)) && exit 5 done print }

# try to copy until success

function trycp { element=“$1” target=“$2” if [[ ! -d ${target:h} ]]; then trymkdir ${target:h} fi local essai=1 print – cp $element $target while ! $element $target; do samelineprint "Echec: essai n°$essai" ((essai++)) ((essai>maxessais)) && exit 5 done print }

# try to remove until success

function tryrm { target=“$1” local essai=1 local options=’’ [[ -d $target ]] && options=‘-rf’ print – rm $options $target while ! rm $options $target; do samelineprint "Echec: essai n°$essai" ((essai++)) ((essai>maxessais)) && exit 5 done essai=1 while [[ -e $element ]]; do samelineprint “rm reussi mais fichier source non disparu n°$essai” sleep 1 ((essai++)) ((essai>maxessais)) && exit 5 done print }

# try to rename until success

function tryrename { element=“$1” target=“$2” local essai=1 while [[ -e $target ]]; do samelineprint “Echec n°$essai le fichier $target existe déjà” ((essai++)) ((essai>maxessais)) && exit 5 sleep 1 done print – mv $element $target while ! mv $element $target; do samelineprint "Echec: essai n°$essai" ((essai++)) ((essai>maxessais)) && exit 4 done essai=1 while [[ -e $element ]]; do samelineprint “mv reussi mais fichier source non disparu n°$essai” sleep 1 ((essai++)) ((essai>maxessais)) && exit 5 done print }

# try to move until success

function trymv { element=“$1” target=“$2” local essai=1 print – mv $element $target while ! mv $element $target; do samelineprint "Echec: essai n°$essai" ((essai++)) ((essai>maxessais)) && exit 5 done essai=1 while [[ -e $element ]]; do samelineprint “mv reussi mais fichier source non disparu n°$essai” sleep 1 ((essai++)) ((essai>maxessais)) && exit 5 done print }

And here is the code on how I synchronize my website. There is a little cryptic code. It correspond a problem caused by the bluecloth filter which is a markdown program made in ruby. Each time my email is written it is transformed differently. This is why I remove this part from the content of each html file. Without it, all my files containing email are different at each regeneration of my website.

`#!/usr/bin/env zsh`

# Script synchronisant le site sur me.com

# normalement, le site est indisponible le moins de temps possible

# le temps de deux renommages de répertoire

# get configuration

# mostly directories

source $0:h/config

# get trycp function (copy until success)

source $0:h/webdav-framework

if [[ $1 == ‘-h’ ]]; then print – “usage : $0:h [-h|-s|-d]” print – " -a sychronise aussi l’index" print – " -h affiche l’aide" print – " -d modification directe (pas de swap)" print – " -s swappe simplement les répertoires" fi

# publication incrementale

function incrementalPublish { local ydestRep=d e s t R e psuffix localRef=“$srcRep/map.yrf” print – “Creation du fichier de references” create-reference-file.sh > $localRef remoteRef="/tmp/remoteSiteMapRef.$$.yrf" if [[ ! -e "$ydestRep/map.yrf" ]]; then # pas de fichier de reference sur la cible print – “pas de fichier de reference sur la cible, passage en mode rsync” rsyncPublish swap else trycp “$ydestRep/map.yrf" "$remoteRef” typeset -U filesToUpdate filesToUpdate=( $(diff $localRef $remoteRef | awk ’/1/ {print $2}' ) ) if ((${#filesToUpdate} == 1)); then print – “Seul le fichier ${filesToUpdate} sera téléversé" elif ((${#filesToUpdate}<10)); then print –”${#filesToUpdate} fichiers seront téléversés :" print -- "${filesToUpdate}" else print – “${#filesToUpdate} fichiers seront téléversés” fi # copy all file with some differences # except the map in case of error for element in $filesToUpdate; do if [[ $element == “/map.yrf” ]]; then continue fi if [[ -e s r c R e pelement ]]; then trycp s r c R e pelement y d e s t R e pelement else tryrm y d e s t R e pelement fi done # if all went fine, copy the map file trycp $srcRep/map.yrf $ydestRep/map.yrf # remove the temporary file }

# publication via rsync

function rsyncPublish { result=1 essai=1 while (( $result > 0 )); do print – rsync -arv $srcRep/ $destRep.tmp if ((!testmode)); then rsync -arv $srcRep/ d e s t R e p.t m p f i r e s u l t=? if (( $result > 0 )); then print -P -- "%BEchec du rsync%b (essai n°$essai)" >&2 fi ((essai++)) done }

# swap

function swap { print -P – “%B[Directory Swap (tmp <=> target)]%b” [[ -e $destRep.old ]] && tryrm $destRep.old

    print -- "  renommage du repertoire sandard vers le .old"
    tryrename $destRep $destRep.old

    print -- "  renommage du repertoire tmp (nouveau) vers le standard"
    print -P -- "%B[Site Indisponible]%b $(date)"
    tryrename $destRep.tmp $destRep
    print -P -- "%B[Site Disponible]%b $(date)"

    print -- "  renommage du repertoire old vers le tmp"
    tryrename $destRep.old $destRep.tmp

    print -P -- "  publication terminée"

}

print – “Root = $webroot” print – “Dest = $destRep”

if [[ “$1” = “-s” ]]; then swap else if [[ “$1” = “-d” ]]; then suffix="" else suffix=“.tmp” fi print -P – “%BSync%b[${Root:t} => d e s t R e p:tsuffix]” incrementalPublish fi

This is my way to replace `rsync` with filesystem not handling it. Hope it is usefull. I’ll be happy to hear a way to handle the webdav rename folder problem. This is really annoying.

1. <>↩

RSS · Tweet · FB
These social sharing links preserve your privacy
↑ Top ↑
Published on 2009-10-28
Follow @yogsototh
Yann Esposito©
Done with Vim spacemacs & nanoc Hakyll

ADA: `DdzFFzCqrhtAvdkmATx5Fm8NPJViDy85ZBw13p4XcNzVzvQg8e3vWLXq23JQWFxPEXK6Kvhaxxe7oJt4VMYHxpA2vtCFiP8fziohN6Yp`