flickrfs

Virtual filesystem for flickr by Manish Rai Jain


Index

About

Flickrfs is a virtual filesystem which mounts on your linux machine like any other partition. Once mounted, it retrieves information about your photos hosted on your flickr account, and shows them as files. You can now easily copy photos from your local machine to this mount, and it will automatically upload them to your flickr account. Similary, you can copy the files from your mount to your local machine, and it will download your images from flickr.

All the files in the mount have a meta file attached to them, which provides access to title, description, tags, and license information. Modifying any of these fields and saving the meta file, will update them on the server as well.

You can easily specify what is the default resolution of photos that you wish to upload, and the fs will automatically resize (read downsize) the photo, and upload it.

That's not all, the latest verion of flickrfs, provides a syncing mechanism. This mechanism automatically syncs any changes done online directly to flickr account (using flickr's web interface or any other third party apps), to your mount. Thus, any addition/deletion/changes of sets/images is reflected back to your mount. No stale information! You just to mount it once, and it can be used forever!

Structure

Flickrfs Home Directory $HOME/.flickrfs:

  • config.txt

    Allows you to set the default image size, and the time interval for the syncing of /sets and /stream to flickr online server.
  • log

    log file which keeps record of activity going on in flickrfs. Useful for debugging purposes. In case you find a bug, post it along with this file on the mailing list.

The filesystem mount point contains these following folders by default:

/sets

/sets folder contains your sets, including your private photos. You can easily add/delete sets using standard linux commands.

/tags (contains /tags/personal and /tags/public)

/tags/personal folder allows search based upon tags in your photostream. Just create a folder (use mkdir) whose name is a colon delimited tags like 'tag1:tag2:tag3', and the fs will search your photostream to match all the photos which contain ALL these tags, and show them as files inside the directory.

/tags/public does the same thing, but searches for public photos even outside your photostream.

/stream (not created by default)

/stream folder will contain ALL the photos that are present and accessible in your photostream. Obviously these wd also include photos from your sets.

This folder is not created by default. You need to create directory named 'stream' in the root of mount point for the fs to populate the photos. Once the folder is created, it can't be removed.

Feature List and Usage

New in flickrfs Release 1.3.9

Integration with Berkeley DB - Minimal hardware resource consumption by flickrfs

flickrfs has been integrated with Berkeley DB using python's bsddb module. Which means flickrfs would now store the image information in Berkeley db database, instead of storing it in memory. Let come millions of images, flickrfs would still consume only negligible amounts of RAM. Regarding hard disk space, it would be in order of 10s of Megabytes; not an issue for today's computers.

Performance increase

flickrfs retrieves the sets information in parallel, background threads. Hence, the directory structure is created quickly, allowing users to start working. If some sets couldn't be retrieved the first time, they'd be taken care of when *syncing* kicks in.

Secondly, full metadata information of images would only be retrieved when asked for; though they'll always be listed in the directory. Its just like the way flickrfs handles images.

Robust execution of flickr operations on small bandwidth

Flickr operations, as in, interactions with flickr server are now fail-safe. flickrfs handles the URLError exceptions thrown by the operation, checks its result, and retries the operation multiple times in case of failure. Thus, even if connection is lost temporarily, flickrfs would continue its operations unaffected.

config file would be automatically created

config.txt file, present in ~/.flickrfs would be created automatically, if not present. Normally the default values would be fine, but feel free to change it according to your needs.

Bandwidth information of flickr account

statfs system call is now tied to the bandwidth usage information of your flickr account. You can view it by running a

$ df -h


New in flickrfs Release 1.2.9

Automatic periodic syncing of fs with online server.

flickrfs will automatically sync any changes done directly through flickr web interface/organizer or any other third party apps, to its local mount dynamically. This include changes to your /sets and /stream. The photos present in /tags are not synced, simply because /tags is meant just for searching/downloading purposes. The time interval can be specified through config.txt file. The default interval specified is 10 mins.


Automatic resizing of photos while uploading.

NOTE: This does not affect the original photo. Only a copy is resized, and uploaded.

If your photo resolution is high, fs will automatically downsize (a copy of it, not original photo) to the resolution specified in config.txt file, before uploading. This also reduces the image's size, thus useful to people with limited 20MB upload limit. Note that if the image is smaller than the specified size, it would not be resized, and the original size will be uploaded.

Unicode Support

More robust unicode support. This feature is still in testing mode, so if you find any bug, feel free to contact us.

 

flickrfs Usage

Edit photo metadata information

Title, Description, Tags, License: Use meta data file associated to photo. For photo named img1_ID.jpg, the meta data file wd be .img1_ID.meta. Modify this file, and save it, and the changes will be updated to the servers.

$ vim .image1_ID.meta

You may also use 'sed' to do batch operations. For e.g., To change the licenses of all photos from license:0 'All rights reserved' to license:1 'Attribution-NonCommercial-ShareAlike License', execute this:

$ sed -i -e 's@license:0@license:1@' .*.meta

Permissions: On the image that you wish to change permissions,execute these statements:

$chmod 755 image_ID.jpg #to make photo public

$chmod 774 image_ID.jpg #to make it available to only friends and family. Not to public.

$chmod 764 image_ID.jpg #to make it available to family only

$chmod 754 image_ID.jpg #to make it available to friends only

$chmod 744 image_ID.jpg #to make it private

Time: Modification time of the image is set to its last update time on flickr, while creation time is set to the time when the image was uploaded to flickr.

$ ls -l #Sort photos in asc. order according to 'last update'

$ ls -cl #Sort photos in asc. order according to 'date uploaded'


Uploading

Copy the image to either /sets/yourset or /stream, with the directoy name as follows:

$ cp /mycamera/myimage.jpg /mntpoint/stream:tag1:tag2:tag3/

and the photo will be uploaded with the three tags, and an additional tag 'flickrfs'. Spaces in tags are allowed.

To create a new set, just create a directory in /sets/. The first photo that you copy or link to that directory will become the primary photo of that set. When you execute a mkdir, copy your first photo quickly, so that the set is created online. Otherwise, the sync thread will delete the new directory, seeing it as incoherent with online server.

$ cp /mycamera/myimage.jpg /mntpoint/sets/newset:tag1:tag2:tag3/

Note the '/' at the end of the destination paths. This is required to differentiate b/w a filename and directory.

Note: /tags/personal can no longer be used for uploading photos. Use /stream for that.

Directly link photos from your stream or tags to sets, without downloading. [Introduced in v1.2.9]

Use your ln command to directly add a photo present in your /stream to a set.

$ ln /mntpt/stream/image.jpg /mntpt/sets/yourset/

$ ln /mntpt/tags/personal/../image.jpg /mntpt/sets/yourset/

OR, if you wish to add a new set altogether, just create a new dir before executing link (ln). When you execute a mkdir, link your first photo quickly, so that the set is created online. Otherwise, the sync thread will delete the new directory, seeing it as incoherent with online server.

$ mkdir /mntpoint/sets/newset


Deleting

Flickrfs doesn't allow/do deletion of photos. It is to prevent a 'rm *' accident!

You can easily remove photos from your sets though.

$ rm /mntpoint/sets/myset/image1_ID.jpg

If the photo is the last photo in the set, the set will be deleted automatically.

Searching

To search your personal photos for tags: tag1, tag2 and tag3; create a directory in /tags/personal:

$ mkdir /mntpoint/tags/personal/tags1:tags2:tags3

To search public photos, do so in /tags/public.

The directory will store the results as files, with a maximum of 500 files.

Downloading

Copy the image from the mount to your local harddisk, and the original size of the image will be downloaded. If original size if not present, the accessible largest size of the photo will be downloaded.

$ cp /mntpoint/<path to image>/image1_ID.jpg ~/myphotos/

Switch Account

If you wish to switch account, just delete the folder $HOMEDIR/.flickr. This will remove your cache, and ask for authentication next time you mount the fs.

$ rm -R ~/.flickr/*

Installation

Ubuntu

1. Open synaptics or use apt-get, and Install the following packages:

  • fuse-utils
  • libfuse2
  • python2.4-fuse (if you have python 2.4 installed)
  • python-fuse
  • Imagemagick (needed for image resizing. You can skip this if you wish to upload the original size of photos only.)

$ sudo apt-get install libfuse2 fuse-utils python2.4-fuse Imagemagick

2. Execute commands:

$ sudo modprobe fuse

$ sudo chmod 755 /usr/bin/fusermount

$ sudo chmod u+s /usr/bin/fusermount

$ sudo chmod 666 /dev/fuse

3. Download flickrfs package from sourceforge, and untar it:

$ tar -xzvf flickrfs-*.tar.gz

OR, at any point of time, the CVS version of flickrfs will be the most updated. So, you can directly download from cvs:

$ cvs -d:pserver:anonymous@flickrfs.cvs.sf.net:/cvsroot/flickrfs login

$ cvs -z3 -d:pserver:anonymous@flickrfs.cvs.sf.net:/cvsroot/flickrfs co -P flickrfs

4. [Optional Step] The first time flickrfs is run, it would automatically create the config.txt file with default values. Otherwise, you can create it yourself, by copying config.txt file in $HOME/.flickrfs. Edit the file, to set the following options:

[configuration]

browser:/usr/bin/firefox

image.size:1024x768

sets.sync.int:600

stream.sync.int:600

The interval is in seconds. The default interval set is 10mins. The sync of /sets and /stream is irrespective of each other, so you can change the interval values as you like. For example, if outdated information in /stream doesn't bother you, you can change the interval to larger values say, 30mins.

image.size field specifies the default size that the photos will be resized to before uploading. If you wish to upload the original size of photos, leave the value blank as follows. Note that, only a copy of photo is resized, your original photo will not be resized.

image.size:

5. [Optional Step] Create a screen terminal

What is screen? Read the Linux Journal article here

Why? This will keep the fs mounted even when you log out. Next time you log in, you can directly access the fs, without having to mount it again. Saves effort!

$ screen -S flickrfs

6. Mount the filesystem

Check if fuse module has already been loaded in kernel.

$ lsmod | grep -i fuse # Rerun commands in step 2 if not loaded.

First time you are mounting the fs, you need to close all the browser windows. Secondly, if you are behind a proxy server, set you http_proxy environment variable in the command line.

$ export http_proxy="http://proxy.address:port"

$ python flickrfs.py /mountpoint

The default web browser used is firefox. If you use something else, you can set it in config.txt.

Your browser window will open up, and require you to login, and allow authentication to flickrfs. Once you do that, close your window. Flickrfs will now start downloading information. Wait a few minutes for your sets to populate. If you wish to view your whole stream, create a directory stream in your mountpoint.

$ mkdir /mntpoint/stream

You are ready to GO!

7. [Optional Step] See flickrfs in action

If you wish, you can view exactly what is flickrfs doing, by checking out the log file present in $HOME/.flickrfs

$ tail -f ~/.flickrfs/log

8. Once you're done, you can unmount the fs

To unmount the filesystem, execute the following command

$ fusermount -u /mntpoint/


Other Linux distros

Follow the installation procedure for ubuntu. The only difference is in step number 1, Installation of packages. You can check if your distro's package manager has these following packages, and install them:

  • fuse
  • fuse-python
  • Imagemagick

If not, you can manually retreive and compile the packages from source.

1. Download and Install FUSE

2. Install fuse python bindings.

$ cvs -d:pserver:anonymous@fuse.cvs.sf.net:/cvsroot/fuse login

$ cvs -z3 -d:pserver:anonymous@fuse.cvs.sf.net:/cvsroot/fuse co -P python

3. This step is optional, only for those who wish to use the image resizing functionality. Check if you already have Imagemagick installed.

$ identify

$ convert

If the above commands produce a 'command not found' error, you need to install Imagemagick.

Download and Install Imagemagick.


Acknowledgements

I would like to thank R. David Murray (flickrfs developer) for his contribution to flickrfs by testing, fixing bugs, and adding new functionalities. I'd also like to thank Febio Spelta (flickrfs developer) for his valuable suggestions, and contributions to flickrfs.

I really appreciate and thank people who provide feedback, even small bugs reports, as these help us to improve flickrfs. And my special thanks to all those who are using flickrfs, and making this whole effort worth.

- Manish Rai Jain

Download latest flickrfs-1.3.9 [New]

(Note that it is recommended to use CVS version rather, as that's the most updated one.) 

Debian versin of flickrfs v1.3.9 as per Mar 27, 2007

flickrfs installation instructions in Portugese. Thanks J. Luiz Coe!

Check out Desktop Flickr Organizer [New]

 

flickrfs on sourceforge

flickrfs on Flickr

Dependencies

flickrfs blogged!

www.flickr.com
More Flickr photos tagged with flickrfs

 

Author's photos:

www.flickr.com
tuxmann's photos tagged with flickrfs More of tuxmann's photos tagged with flickrfs