Using find,sed and xargs to strip files from one folder that exist in another

Handy bash commands to remove copies of files that exist in two places when you only want unique files in each folder:

Scenario:
Folder A contains 20000 files
Folder B contains 50000 fies

… and you have accidentally copied all of folder A into folder B so you now have thousands of duplicate files in folder B.

You can easily strip out any files in Folder B from Folder A using 3 bash commands piped together like so:

find foldera/* | sed s@foldera/@folderb/@ | xargs rm -rf

  • find – locate all files in folder a (you can use any combination of find paramaters here of course)
  • sed – replace foldera with folder b in each line of the results
  • xargs – run the rm -rf command on the output from sed
This just save me a lot of  time. Hope it helps someone else.
Credits to CiaranG for the assist with piping find results into sed, nice work!

Sync filezilla site manager across all your machines with Dropbox

Another timesaver here, if you are like me and use filezilla client on various different machines and platforms this is a great time saver, we are going to share the sitemanager.xml file that filezilla uses to store all your site definitions on different machines. Personally I have over 200 ftp connections and constantly working between office, home and mobile keeping my ftp client current is a total mare. Luckily dropbox and someeasy steps solve all that.

First things first grab a copy of your Filezilla sitemanager.xml and stick it in a dropbox folder, i use:

Dropbox/Apps/Filezilla

Just take it from whichever machine is most uptodate, for linux it will be in ~/.filezilla/sitemanager.xml, for Windows 7 it will be here c:\Users\Username\AppData\Roaming\Filezilla\sitemanager.xml

Once you’ve done this all you need to do is create symobilic links on each machine pointing at the sitemanager.xml file the dropbox folder.

Linux
Surprise, surprise this is the easiest of the lot. Just do this (obviously use your dropbox path if different)

ln -s ~/Dropbox/Apps/Filefillz/sitemanager.xml ~/.filezilla/sitemanager.xml

Windows 7
This took a while to sort out, there is a command called mklink on win7 that can do links but i gave up after a while as i kept getting syntax errors. Then i found this little shell extension which works great:

http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html

Basically once you’ve installed the dll then the shell extension you can simply locate the target file (the dropbox one) right click and choose ‘Pick Link Source’

Then just browse to the folder where your sitemanager.xml should be stored (mine is in C:\Users\david\AppData\Roaming\FileZilla) then right click in the whitespace and click Drop

Mac

Not on this blog in my lifetime, get a grip….

Summary

Follow the process above on all your machines and then just open up filezilla and you should see all your site manager settings, try adding/editing one and then opening filezilla  you should see it sync onto your other machines immediately.

Also Ive noticed if you update the sitemanager, you dont need to restart filezilla, each time you click the sitemanager icon it will load a fresh copy of the xml file, smooth!

Searching folder of files for a string

Ill never remember this and its just come in very handy

grep -lr “string” path

-l – this makes it list the filenames where it finds the string
-r – this makes it recursively scan all sub folders

e.g.

grep -lr “/var/www/proj” ./*

This would scan for the string “var/www/prog” in all files/folders starting from the current working directory.

Love linux!

 

http://old.personalprojector.co.uk/catalog/admin/