Hidden files—also known as dot-files–are a type of file or directory that saves configuration settings or important data required for a program to function properly. These files begin with a . character which makes them invisible. To view them, you can run the command ls -a. Most hidden files are saved in home directories, but can be found elsewhere. Newer users tend not to make changes to these files but as they develop their own preferences or when specific needs arise, it may become desierable to do so.
To give you an idea of the kinds of modifications you can make, let’s discuss a few examples of some of the more commonly used hidden files.
Run Command Files
One of the most well-known “Run Command” (“rc”) files is the .bashrc file. As the name implies, these files will run commands at start up and are typically used to create a custom environment with personal preferences enabled.
Every time users log in to the Discovery or Endeavour cluster, the commands saved in .bashrc are run. For example, if you wanted to customize your command prompt or set an alias you could add them to this file.
Changes made here take effect every time you log in so if you aren’t careful, you could have trouble with subsequent logins.
Other “rc” files include:
| File name | Comments |
|---|---|
.cshrc |
Settings for c-shell users |
.screenrc |
Settings for screen
|
.vimrc |
Settings for vim text editor |
.pythonrc |
When used in conjuction with the PYTHONSTARTUP environment variable, this runs commands in interactive startup |
~./R/makevars |
Used to set up build environment for R package installs |
Other Kinds of Hidden Files
History files
History files such as .bash_history or .Rhistory keep a record of previous commands that you have run so you can refer back to them later if needed.
Configuration files
Configuration files allow you to override the default behavior of a program so you can customize it for your needs. For example, .gitconfig allows you to specify the default email and username to associate code changes with when you run git commit.
Temporaray data files
Some programs will create data files to store important information while running. For example, when editing a file with vim, data is periodically written to a .swp file. If something goes wrong before you can save, vim will attempt to recover unsaved changes from this file.
Hidden Directories
Hidden directories are normally application-specific directories that contain configuration settings or data required for the application to function properly. Let’s go over a few that you may encounter on Discovery. These directories can sometimes take up a lot of disk space so if you encounter disk quota issues, this may be a good place to check for ways to free up data.
~/.conda
If you have such a directory, you should see a file named environments.txt, and two directories named envs and pkgs.
The environments.txt file is a list of paths for every anaconda environment you have created. Your environments are saved in the envs directory. Individual packages and package metadata are saved in pkgs.
~/.cache
Files saved here are often downloaded and kept here to prevent a second re-download.
For example, when using pip to install a python package, a copy of the package is saved to ~/.cache/pip. If something goes wrong during the installation process, pip will used the saved package in ~/.cache to save time.
~/.local
This directory is used to save “local” or user-level installations of software. On Discovery, the most common use for this directory is for user-installed python packages. If you have this directory, you should see bin, lib, and share subdirectories.
Now that you know about some of the hidden files you may encounter on Discovery/Endeavour, I would like to encourage you to take a look at your own files and even make some customizations to your environment. If you do decide to make changes, it’s recommended that you create a copy of the original file first, so you can easily revert changes. In the event that you somehow accidentally lock yourself out of your account due to changes you’ve made, you can send an email to carc-support@usc.edu and we can temporarily revert the file for you.