My Use Case:

I try basicly to use annex as a raid-like tool (at least thats the first step)

so I added some big files to it, and then I synced it to a usb-remote. So far all did go well...

No I startet annex watch daemon... because I thought it would then watch the files and checkin and out at least for the "origin" copy of the files.

Then I thought lets delete some files I dont need anymore... so... rm.... ^^

Maybe I just wanted to see what happens or if that would magicly do what I wanted him to do... ok I knew that it would not delete the file on the usb-drive (backup...) ok... but maybe at least localy...

Now what did happen instead of that... the links are gone yes... the files it self are in the .git objects tree, so they did not get deleted, so ok not the way I wanted... have to unlock it first, would make maybe sense...

So I tried first to get the links back... tried fix, tried unused, tried get... but the links doesnt show up again...

is there a way to first bring back the links? is it save or the right way to just git rebase HEAD~3 to bring the links back?

and then when I want to delete files from all places whats the way to do that... annex unlock -> then delete? or git drop filex --copies=0 or something?

like I said I try to use is like a more flexible raid thing, this files are to big to really back em up with history... I watch them... and then soon I will often delete them (from everywhere). but other parts stay... I dont delete them then... ^^

So maybe I missuse annex for that usecase... try to find that out ;)

You can use any git command you like like git rebase or git revert to operate on the links.

git-annex, like git, keeps data for files that have been deleted.

The git annex unused command will find that data and let you remove it. See unused data.

Comment by http://joeyh.name/ Sun Sep 16 00:14:49 2012

git-annex version: 3.20120629

git annex unused did not show me anything... dont know why... I did only "rm Directory" not git rm... but "git annex watch" did check it in... I sadly have reverted and droped the file now... but if you want I can try to delete other files the same way to see if annex unused shows them... then I post the hole commands... k I try it now:

oh I see the directory I wanted to drop did not get dropped because I set numcopies to 2 ;) so I try it again with that file:

  1. rm -r directory
  2. ls it is gone (should be ;) )

user@host:~/annex/SOFTWARE$ git annex unused unused . (checking for unused data...) (checking master...) (checking synced/master...) (checking usbdrive/master...) ok user@host:~/annex/SOFTWARE$

Is that like it should be? If so, should unused only show stuff that I delete with git rm?


Ok I did read the unused doku you pointed me. So I thought I got it, but I did not:

If I

  1. rm File
  2. in the usb-repos, git pull both repositories dont have a link to the file.
  3. git unused shows nothing
  4. it seemed that the link AND the File it links to got deleted:

-../.git/annex/objects/pM/vq/SHA256-s1066518528--00c5e1b1610c0c2dfab05c7a55aaad

but the file size is 1gb, both directories dont get smaller... also if I use git gc after that it does not get smaller too.

Is there a way to really delete files from both repositories.

unused . (checking for unused data...) (checking master...) (checking synced /master...) (checking usbdrive/master...) ok

It still considers a file's content used while some branch contains the file. So one of the branches it's checking there still has your file in it. Once you get all the branches synced up, the file will become unused.

Comment by http://joeyh.name/ Sun Sep 16 17:08:49 2012

hmm Ok I tried now something that seems to did it... after I did checkin and pull the deletion of the soft-link of the file... I addionaly used git annex sync, and then again pushed it from the usb drive... now the unused command finds stuff...

So am I right now to make that happen:

  1. delete the file in the "main" repos
  2. git pull from the other repository(s)
  3. git annex sync

  4. git annex unused (now it founds something to delete)

  5. git annex dropunused 1-1000 (or some other range) do that in both repositories to delete in on both... and do that maybe as cronjob (from point 2 to 5) or do the last last step only if a drive is nearly full (automaticly or manually...)

ok so maybe I could use that.

the advantages over something like rsync are...

  1. I have a history and can reverse some operations (till I do step 5 ^^)
  2. I am more flexible in repo/remote types
  3. checksums get saved
  4. I can have repo with all links but 0 data on a 3rd pc and can "get" stuff dynamicly (what is the cheapest transfeir) ot them... aka partial checkouts

ok I think the 4. advantage buys me ;)

hope I got it now right with the 5 step instruction.

keep up your great work, would have tried kickstarter here maybe too for other stuff... but I am no american ;) and eu-guys are not allowed, and I also have no relative in usa ;) but that only as a site note ;)

That looks right except you shouldn't have to git pull from the other repository(s). The first thing git annex sync does is a git pull.

ok I now at least got it that it is nice to run watch on both repositories...

but what I dont get is, is the git annex watch commando not a automatic way so that this deamon makes sync command whenever needed? so I have to use git annex sync first on the repos where I delete the file and then again on the repos I want to receive that change... that seemed to work, but why did watch not do that for me???

cd /mnt/data/annex/ && /usr/bin/git annex sync && /usr/bin/git annex copy --auto --to usbdrive && cd /mnt/usb1/annex/ && /usr/bin/git annex sync && echo "synced" || echo "backup of annex failed" | sendxmpp -t name@server.country

ok suprisingly that did not only sync the files but also did remove the file, so you dont need a pull... dont know what I did wrong the other times I tried, does maybe the copy command also "copies" the delete commit?

but annyway that works for me... (for now I do the dropunused command manuelly...)

that should work as cronjob...

Comments on this page are closed.