Recovering Files

Sometimes, you may have deleted your files accidentally and you want to recover them.

Important Note: Always double check before you are deleting your directories. The files in /home directory are only protected by snapshots at this moment. Please exercise more than typical care when removing files from /home. We will notify you once the backup is fully functional again.

On HPC Storage system

/home directory

You can retrieve the files from 7 days snapshots residing at your home directory /home/<CaseID>/.snapshot. The snapshot process initiates at 11pm nightly, and takes some time (it is not instantaneous to compare changes to the reference). Note that we can not recover files that are created and deleted within a short period that does not all it to be captured in the "overnight" snapshot process.

Snapshot:

ls /home/<caseID>/.snapshot

output:

2014.02.04.23.00.01.home 2014.02.05.23.00.01.home 2014.02.06.23.00.01.home 2014.02.07.23.00.01.home 2014.02.08.23.00.01.home 2014.02.09.23.00.01.home 2014.02.10.23.00.01.home

(Note: The '.snapshot' will not tab-complete, it must be typed directly.)

Recover

Copy back your directory to your home directory. If the data size is huge, consider Globus transfer.

cp -r <timestamp>.home/<path-to-directory-to-be-restored> .

Example: Copying all hidden files from the snapshot (preserving the source permission)

cp -rp /home/<caseID>/.snapshot/<time-stamp>.home/.[^.]* .

Want to do a "diff" between 2 snapshot folders?

diff -bur default_<time_stamp>/ default_<time_stamp>/

/mnt/pan/<ID> Directory:

Panasas storage can be purchased with snapshot options and the data can also be archived for additional cost (see Performance Storage & Archival). Find the snapshot (.snapshot) directory at /mnt/pan/<your-allocated-directory).

cd /mnt/pan/<your-allocated-directory)/.snapshots

/scratch

If you have deleted the files from your home directory accidentally, you may also find them in the scratch space if it has not crossed 14 days expiry time. You can copy them in your home directory

cp -r /scratch/pbsjobs/job.<jobID>.hpc/ .

On Vstor

A lot of data stored on the Research Storage Vstor is protected by the "snapshot" process. To recover files or directories, cd close to the directory of interest as possible. For example, if a directory "example" is missing in the path /mnt/vstor/example/, check the directory:

ls /mnt/vstor/example

archive/ code/ data/ papers/

You know that the directory "papers" was deleted from there. Now, CD to that directory and then to .snapshots

cd /mnt/vstor/example

cd .snapshot

Check the time stamps:

...

Testing_2023-12-03_17_30_33_UTC

Testing_2023-12-04_17_30_33_UTC

Testing_2023-12-05_17_30_33_UTC

Now list the contents of these 'snapshot' directories to find the missing directory from the appropriate time stamp.

ls Testing_2023-12-05_17_30_33_UTC

papers/ ...

To recover, perform 'cp' into your working directory.

cp -r Testing_2023-12-05_17_30_33_UTC/papers /mnt/vstor/example/

Make sure that the papers directory is copied.

ls /mnt/vstor/example

... papers/