Migrating a legacy Discovery home directory to DartFS

Background

New home directories for Research Computing servers are in DartFS.  When you have a DartFS home directory you can login to all of our systems (Discovery, Polaris, Andes, etc.) using your NetID and you have the same home directory everywhere.  This document describes how to move data from an old home directory (i.e. one that you logged into with a username that is not your NetID) to DartFS. 

Step by step

  1. Request a DartFS home at https://rcweb.dartmouth.edu/accounts.
  2. Wait for an email confirming that the new home is ready.  Be sure to check your Junk folder as well. DartFS accounts are usually provisioned within 48 hours.
  3. Make sure you know your old username and password.  Email research.computing@dartmouth.edu if you need your password reset.
  4. Find a few hours when you won't be running jobs on Discovery.  You want to be sure that your legacy home is quiescent so that everything copies correctly.
  5. Login to discovery7 using your OLD username.
    • ssh username@discovery7.dartmouth.edu
  6. Run the migrate_home script.  The script will prompt you for your NetID and password and then copy everything from your legacy home directory to the Discovery_home folder in your DartFS home.
    • migrate_home
  7. Logout and wait for an email announcing the copy has completed.  Again, check your Junk folder if you don't see it within a few hours.
  8. Make sure everything looks good with the copy.  You should pay particular attention if the email says there were problems.  Those will be logged in a file that begins with rsync_Discovery_home... and ends with .error
    • ssh netid@discovery7.dartmouth.edu
    • cat rsync_Discovery_home*.error

Troubleshooting

By default, DartFS homes are private to you.  If you had colleagues who accessed data from your legacy home directory then that sharing will have to be reimplemented in DartFS.  We can help with that.

Very few errors are fatal to the copy process.   For the most part the script will copy everything it can, skipping (and recording to the error log) only files that it cannot copy for whatever reason.  If you have errors, we can help with these too.  Some examples.

  • If you have files in your home directory that are not readable by you then those obviously cannot be copied.
  • If you have a directory with the "sticky" bit set ('t' or 'T' in the last column of the unix permission bits), files in that directory will be copied but will have a random extension appended to the name.  For example, file.txt might be file.txt.Mn4.
  • If you already have data in your DartFS home then there may not be room to copy everything.  Use the quota command to check usage.
  • So called "sparse" files can expand greatly when copied.  On rare occasions this might also mean the destination ends up not having enough space for the copy.

Email us at research.computing@dartmouth.edu with any questions or problems.

Print Article