Command Line Intro

Today we’re going to dig a little deeper into using the command line, focusing on some of the most useful commands that you’ll likely use in day-to-day development. We’ll specifically look at working with files and searching.

Goal for this Tutorial:

Create, move, and remove files and directories.
Understand file permissions on a Unix system.
Learn to install new tools on Ubuntu/Debian Linux.
Learn how to search within file and by file name.

We learned in how to move around with in a terminal in the previous tutorial, and today we’re going to focus on manipulating files.

Files are the core of the computer from the user’s view. We put our source code in files, our executables are files, and images and other data formats are all files. Because of this, being confident when working with files is very important to how we operate as programmers.

There are some differences between how some commands work between macOS and linux, so for this tutorial I’m going to work from within a Linux container. If you have Docker installed and have a docker server that you can connect to, run this command to follow along:

$ docker run --rm -it  -w /usr/src ubuntu:16.04 bash

Creating Files and Understanding Permissions

To learn how file’s work we’re going to cover how to create, copy, move, and delete them. Let’s start with creating files.

Creating files on a unix system can happen in a few different ways. You can create a file by writing it from your text editor, if the file doesn’t exist it will be created when you save your changes. The other way that we create files is by using the touch command.

$ touch example.txt

Now if we use ls -al we should see the new file:

$ ls -al
total 8
drwxr-xr-x  2 root root 4096 Oct 29 16:36 .
drwxr-xr-x 12 root root 4096 Oct 29 16:36 ..
-rw-r--r--  1 root root    0 Oct 29 16:36 example.txt

Let’s break down that first section of our line for example.txt:

-rw-r--r--

The first dash represents whether or not the item is a file or directory. If example.txt were a directory we would see a d there like we do for the .. line.

Positions 2-4 represent the permissions for the owner of the file. Positions 5-7 represent the permission for the file’s group members. Lastly, positions 7-9 represent the permissions for anyone on the system. All of these sections work the same way, but they represent access for a different level of user.

The three positions in each of the groupings represent:

read
write
execute

Knowing that we can see that the root user (that’s us in this container) can read and write the file, anyone in the root group can read the file, and anyone on the system can read the file.

There are tools specifically for changing file permissions and ownership. I’m going to list them out with a brief description, but I encourage you to look them up using man:

chmod - Change the file’s permissions.
chown - Change the file’s owner.
chgrp - Change the file’s group.

Moving, Copying, and Removing Files

Using our example.txt file, we can go over the rest of the common actions that you’ll perform on files, but before we do that we’re going to create a new directory to move it to.

mkdir is the tool that you’ll use to make directories.

$ mkdir -p test/examples

That command actually creates two directories for us. If the test directory doesn’t exist it will create it, and then it will also create an examples directory underneath of test.

Now we can move our file to one of these different directories. When moving a file sometimes what we actually want to do is create a copy of the file in a different spot. Let’s do that now using cp.

$ cp example.txt test/examples
$ ls test/examples
example.txt
$ ls
example.txt test

You can see that we now have an example.txt in the test/examples directory, but what would it look like if we wanted to move a file completely? For that we’ll use the mv utility.

$ mv test/examples/example.txt test/sample.txt
$ ls test
examples sample.txt
$ ls test/examples
$

That completely moved the file from test/examples/example.txt to be at test/sample.txt. In the process we also renamed the file. The mv tool is also the tool that you would use if you want to rename a file in the same directory.

We no longer need the test directory or files in it so let’s look at how we would get rid of those using rm.

$ rm test/sample.txt
$ ls test
examples
$ rm test
rm: cannot remove 'test': Is a directory

rm is used for removing files by default. It is also the tool that we’ll use to remove directories, but we’ll have to provide it with a specific flag, -r (for recursive).

$ rm -r test
$ ls
example.txt

The recursive flag means that it will also remove the contents of the directory. We no longer have either test or test/examples.

Searching and Finding Files

Knowing how to manipulate files will allow you to perform the file maintenance that you’ll need to do from day-to-day, but one of the more useful things that you’ll need to be able to do is search for files by name and by content. We’ll use the tools find and grep respectively to accomplish these goals.

Before we can search through files we’re going to need to have something of substance to search through. For that we’re going to pull down the rails application that we worked with before Hours, but we can’t easily get that into our container without installing some additional tools.

Installing Packages on Ubuntu/Debian

We’re going to use the built-in package manager for Ubuntu and Debian to install wget so that we can download the zip archive of the Hours project.

$ apt-get update -yqq && apt-get install -yqq wget

This line actually does quite a bit. We’re first updating the list of known packages on our system so that it knows what we can download and install. From there we use && to chain on another command, this time we’re installing the wget command. In both of these commands we’re passing the -yqq flag to suppress some of the output that these commands give (making them more “quiet”) and we’re also saying “yes we want to perform this action”. Without the -y we would need to type y again to allow the command to complete.

Downloading and Unzipping Hours

With wget installed we can grab the zip link from github to download the repo.

$ wget https://github.com/DefactoSoftware/Hours/archive/development.zip

Now we have a development.zip directory that we can unzip using the pre-installed unzip command.

$ unzip development.zip -d Hours

That will give us a sufficiently big project to search through.

Finding Files by Name

If you are working with a codebase that you’re not very familiar with it can be helpful to search for files that contain specific words as a way of exploring. We’ll do this now by looking for file names that contain the word “secret”

$ find . -name "*secret*"
./Hours-development/config/initializers/secret_token.rb

The . after find indicates that we want to look at all files under our current directory. By passing the -name flag we’re stating that we want to search by name. The token "*secret*" means we want to look for any file name including secret and we don’t care what characters come before or after it. The * wildcard character is very helpful when using find, otherwise we would be looking for a file that’s entire name was “secret”.

Searching File Contents

Admittedly, you’ll search for files by name every now and then, but you’ll search the contents of files all the time if you’re anything like me.

It’s amazing how often I need to see where a specific class or method is used in a code base. For that we use tools like grep. Let’s see where in the codebase the Tag model is used.

$ grep -Rn "Tag" .

There are quite a few things that match Tag there so I won’t put all of the contents here, but some of the matches are for things unrelated to our model. For this we’ll make our search a little more specific.

$ grep -Rn "Tag\." .

This search looks for anytime we use Tag explicitly followed by a .. If we didn’t escape the period with a \ then wouldn’t work the way we expect because . is a wildcard character when searching with grep.

Breaking this command down all of the way we’re using the -R flag to search in all directories recursively. The -n flag tells grep to return the line numbers where the matches are found in the files. Finally, the . at the end is the path for grep to search. We could be more specific with the directory to further fine tune our output. For instance we could exclude the test results by using Hours-development/app as the directory to search.

Recap

Now that you know the basics of working with and searching files you should feel much more comfortable in the command line. Between working with files and understanding how the shell reads command and where commands come from you’re in a good spot to use the terminal and command line as part of your normal development tool chain.

Let me know if there are any other command line specific topics you’d like me to cover, if not I’ll likely cover new things in passing as we use them in tutorials for other topics.

Command Line Intro – Part 2