Hidden files—also known as dot-files–are a type of file or directory that saves configuration settings or important data required for a program to function properly. These files begin with a
. character which makes them invisible. To view them, you can run the command
ls -a. Most hidden files are saved in home directories, but can be found elsewhere. Newer users tend not to make changes to these files but as they develop their own preferences or when specific needs arise, it may become desierable to do so.
To give you an idea of the kinds of modifications you can make, let’s discuss a few examples of some of the more commonly used hidden files.
One of the most well-known “Run Command” (“rc”) files is the
.bashrc file. As the name implies, these files will run commands at start up and are typically used to create a custom environment with personal preferences enabled.
Every time users log in to the Discovery or Endeavour cluster, the commands saved in
.bashrc are run. For example, if you wanted to customize your command prompt or set an alias you could add them to this file.
Changes made here take effect every time you log in so if you aren’t careful, you could have trouble with subsequent logins.
Other “rc” files include:
||Settings for c-shell users|
||When used in conjuction with the
||Used to set up build environment for R package installs|
History files such as
.Rhistory keep a record of previous commands that you have run so you can refer back to them later if needed.
Configuration files allow you to override the default behavior of a program so you can customize it for your needs. For example,
.gitconfig allows you to specify the default email and username to associate code changes with when you run
Some programs will create data files to store important information while running. For example, when editing a file with
vim, data is periodically written to a
.swp file. If something goes wrong before you can save,
vim will attempt to recover unsaved changes from this file.
Hidden directories are normally application-specific directories that contain configuration settings or data required for the application to function properly. Let’s go over a few that you may encounter on Discovery. These directories can sometimes take up a lot of disk space so if you encounter disk quota issues, this may be a good place to check for ways to free up data.
If you have such a directory, you should see a file named
environments.txt, and two directories named
environments.txt file is a list of paths for every anaconda environment you have created. Your environments are saved in the
envs directory. Individual packages and package metadata are saved in
Files saved here are often downloaded and kept here to prevent a second re-download.
For example, when using
pip to install a python package, a copy of the package is saved to
~/.cache/pip. If something goes wrong during the installation process,
pip will used the saved package in
~/.cache to save time.
This directory is used to save “local” or user-level installations of software. On Discovery, the most common use for this directory is for user-installed python packages. If you have this directory, you should see
Now that you know about some of the hidden files you may encounter on Discovery/Endeavour, I would like to encourage you to take a look at your own files and even make some customizations to your environment. If you do decide to make changes, it’s recommended that you create a copy of the original file first, so you can easily revert changes. In the event that you somehow accidentally lock yourself out of your account due to changes you’ve made, you can send an email to email@example.com and we can temporarily revert the file for you.