Today we’re going to dig a little deeper into using the command line, focusing on some of the most useful commands that you’ll likely use in day-to-day development. We’ll specifically look at working with files and searching.
Goal for this Tutorial:
- Create, move, and remove files and directories.
- Understand file permissions on a Unix system.
- Learn to install new tools on Ubuntu/Debian Linux.
- Learn how to search within file and by file name.
We learned in how to move around with in a terminal in the previous tutorial, and today we’re going to focus on manipulating files.
Files are the core of the computer from the user’s view. We put our source code in files, our executables are files, and images and other data formats are all files. Because of this, being confident when working with files is very important to how we operate as programmers.
There are some differences between how some commands work between macOS and linux, so for this tutorial I’m going to work from within a Linux container. If you have Docker installed and have a docker server that you can connect to, run this command to follow along:
$ docker run --rm -it -w /usr/src ubuntu:16.04 bash
Creating Files and Understanding Permissions
To learn how file’s work we’re going to cover how to create, copy, move, and delete them. Let’s start with creating files.
Creating files on a unix system can happen in a few different ways. You can create a file by writing it from your text editor, if the file doesn’t exist it will be created when you save your changes. The other way that we create files is by using the
$ touch example.txt
Now if we use
ls -al we should see the new file:
$ ls -al total 8 drwxr-xr-x 2 root root 4096 Oct 29 16:36 . drwxr-xr-x 12 root root 4096 Oct 29 16:36 .. -rw-r--r-- 1 root root 0 Oct 29 16:36 example.txt
Let’s break down that first section of our line for
The first dash represents whether or not the item is a file or directory. If
example.txt were a directory we would see a
d there like we do for the
Positions 2-4 represent the permissions for the owner of the file. Positions 5-7 represent the permission for the file’s group members. Lastly, positions 7-9 represent the permissions for anyone on the system. All of these sections work the same way, but they represent access for a different level of user.
The three positions in each of the groupings represent:
Knowing that we can see that the
root user (that’s us in this container) can read and write the file, anyone in the
root group can read the file, and anyone on the system can read the file.
There are tools specifically for changing file permissions and ownership. I’m going to list them out with a brief description, but I encourage you to look them up using
chmod- Change the file’s permissions.
chown- Change the file’s owner.
chgrp- Change the file’s group.
Moving, Copying, and Removing Files
example.txt file, we can go over the rest of the common actions that you’ll perform on files, but before we do that we’re going to create a new directory to move it to.
mkdir is the tool that you’ll use to make directories.
$ mkdir -p test/examples
That command actually creates two directories for us. If the
test directory doesn’t exist it will create it, and then it will also create an
examples directory underneath of
Now we can move our file to one of these different directories. When moving a file sometimes what we actually want to do is create a copy of the file in a different spot. Let’s do that now using
$ cp example.txt test/examples $ ls test/examples example.txt $ ls example.txt test
You can see that we now have an
example.txt in the
test/examples directory, but what would it look like if we wanted to move a file completely? For that we’ll use the
$ mv test/examples/example.txt test/sample.txt $ ls test examples sample.txt $ ls test/examples $
That completely moved the file from
test/examples/example.txt to be at
test/sample.txt. In the process we also renamed the file. The
mv tool is also the tool that you would use if you want to rename a file in the same directory.
We no longer need the
test directory or files in it so let’s look at how we would get rid of those using
$ rm test/sample.txt $ ls test examples $ rm test rm: cannot remove 'test': Is a directory
rm is used for removing files by default. It is also the tool that we’ll use to remove directories, but we’ll have to provide it with a specific flag,
-r (for recursive).
$ rm -r test $ ls example.txt
The recursive flag means that it will also remove the contents of the directory. We no longer have either
Searching and Finding Files
Knowing how to manipulate files will allow you to perform the file maintenance that you’ll need to do from day-to-day, but one of the more useful things that you’ll need to be able to do is search for files by name and by content. We’ll use the tools
grep respectively to accomplish these goals.
Before we can search through files we’re going to need to have something of substance to search through. For that we’re going to pull down the rails application that we worked with before Hours, but we can’t easily get that into our container without installing some additional tools.
Installing Packages on Ubuntu/Debian
We’re going to use the built-in package manager for Ubuntu and Debian to install
wget so that we can download the zip archive of the Hours project.
$ apt-get update -yqq && apt-get install -yqq wget
This line actually does quite a bit. We’re first updating the list of known packages on our system so that it knows what we can download and install. From there we use
&& to chain on another command, this time we’re installing the
wget command. In both of these commands we’re passing the
-yqq flag to suppress some of the output that these commands give (making them more “quiet”) and we’re also saying “yes we want to perform this action”. Without the
-y we would need to type
y again to allow the command to complete.
Downloading and Unzipping Hours
wget installed we can grab the zip link from github to download the repo.
$ wget https://github.com/DefactoSoftware/Hours/archive/development.zip
Now we have a
development.zip directory that we can unzip using the pre-installed
$ unzip development.zip -d Hours
That will give us a sufficiently big project to search through.
Finding Files by Name
If you are working with a codebase that you’re not very familiar with it can be helpful to search for files that contain specific words as a way of exploring. We’ll do this now by looking for file names that contain the word “secret”
$ find . -name "*secret*" ./Hours-development/config/initializers/secret_token.rb
find indicates that we want to look at all files under our current directory. By passing the
-name flag we’re stating that we want to search by name. The token
"*secret*" means we want to look for any file name including
secret and we don’t care what characters come before or after it. The
* wildcard character is very helpful when using find, otherwise we would be looking for a file that’s entire name was “secret”.
Searching File Contents
Admittedly, you’ll search for files by name every now and then, but you’ll search the contents of files all the time if you’re anything like me.
It’s amazing how often I need to see where a specific class or method is used in a code base. For that we use tools like
grep. Let’s see where in the codebase the
Tag model is used.
$ grep -Rn "Tag" .
There are quite a few things that match
Tag there so I won’t put all of the contents here, but some of the matches are for things unrelated to our model. For this we’ll make our search a little more specific.
$ grep -Rn "Tag\." .
This search looks for anytime we use
Tag explicitly followed by a
.. If we didn’t escape the period with a \ then wouldn’t work the way we expect because
. is a wildcard character when searching with grep.
Breaking this command down all of the way we’re using the
-R flag to search in all directories recursively. The
-n flag tells grep to return the line numbers where the matches are found in the files. Finally, the
. at the end is the path for grep to search. We could be more specific with the directory to further fine tune our output. For instance we could exclude the test results by using
Hours-development/app as the directory to search.
Now that you know the basics of working with and searching files you should feel much more comfortable in the command line. Between working with files and understanding how the shell reads command and where commands come from you’re in a good spot to use the terminal and command line as part of your normal development tool chain.
Let me know if there are any other command line specific topics you’d like me to cover, if not I’ll likely cover new things in passing as we use them in tutorials for other topics.