2023/3/2 01:09 Lab 2: Shell
The shell is the main command-line interface between a user and the operating system, and it is an essential part of the daily lives of computer scientists, software engineers, system administrators, and such. It makes heavy use of many OS features. In this lab, you will build a simpli ed version of the Unix shell called the New Yet Usable SHell, or for short.
Please review the rst lecture of MIT’s The Missing Semester of Your CS Education if you are not familiar with the shell.
Through this lab, you will:
Familiarize yourself with the Linux programming environment and the shell, of course.
Learn how to write an interactive command-line program.
Learn how processes are created, destroyed, and managed.
Learn how to handle signals and I/O redirection.
Get a better understanding of the OS and system calls.
Be a better C programmer and be better prepared for your future technical job interviews. In particular, the string parsing skill that you will practice in this lab is desired in many interview questions.
The shell is essentially a command-line interpreter. It works as follows:
1. It prompts you to enter a command.
2. It interprets the command you entered.
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
sevitcejbO
noitcudortnI llehS :2 baL
2023/3/2 01:09 Lab 2: Shell
If you entered a built-in command (e.g., cd ), then the shell runs that command.
If you entered an external program (e.g., ), or multiple programs connected through pipes (e.g., ), then the shell creates child processes, executes these programs, and waits for all these processes to either terminate or be suspended.
If you entered something wrong, then the shell prints an error
3. Rinse and repeat until you press to close or enter the
built-in command exit , at which point the shell exits.
Your shell should follow these speci cations carefully. These speci cations may be slight different from the default Linux shell ( bash ) for simplicity.
The prompt is what the shell prints before waiting for you to enter a command. In this lab, your prompt must have exactly the following format:
An opening bracket [ .
The word .
A whitespace.
The basename of the current working directory. A closing bracket ] .
A dollar sign $ . Another whitespace.
For example, if you are in , then the prompt should be:
If you are in the root directory ( / ), then the prompt should be:
ls -l | less
/home/abc123/2250/lab2
[nyush lab2]$ █
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
tpmorp ehT
snoitac icepS
2023/3/2 01:09 Lab 2: Shell
Note that the nal █ character in these examples represents your cursor; you should not print that character in your shell prompt.
Keep in mind that is line-buffered by default. Therefore, don’t forget to ush immediately after you print the prompt. Otherwise, your program may not work correctly with the autograder.
In each iteration, the user inputs a command terminated by the “enter” key (i.e., newline). A command may contain multiple programs separated by the pipe ( | ) symbol. A valid command must satisfy the following requirements:
If there are multiple programs in a command, only the rst program may redirect its input (using < ), and only the last program may redirect its output (using > or >> ). If there is only one program in a command, it may redirect both input and output.
In each command, there are no more than one input redirection and one output redirection.
Built-in commands (e.g., cd ) cannot be I/O redirected or piped.
For simplicity, our test cases have the following assumptions:
Each command has no more than 1000 characters.
There is always a single space separating lenames, arguments, and the pipe and redirection symbols ( | , < , > , >> ).
There are no spaces within a lename or an argument.
For your reference, here is the grammar for valid commands (don’t worry if you can’t understand it; just look at the examples below):
[nyush /]$ █
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
dnammoc ehT
CS Help, Email: tutorcs@163.com
2023/3/2 01:09 Lab 2: Shell
Here are some examples of valid commands: A blank line.
/usr/bin/ls -a -l
cat shell.c | grep main | less
cat < input.txt
cat > output.txt
cat >> output.txt
cat < input.txt > output.txt
cat < input.txt >> output.txt
cat > output.txt < input.txt
cat >> output.txt < input.txt
cat < input.txt | cat > output.txt
cat < input.txt | cat | cat >> output.txt
[command] := “”; or
:= [cd] [arg]; or
:= [exit]; or
:= [fg] [arg]; or
:= [jobs]; or
:= [cmd] ‘<' [filename] [recursive]; or
:= [cmd] '<' [filename] [terminate]; or
:= [cmd] [recursive]; or
:= [cmd] [terminate] < [filename]; or
:= [cmd] [terminate].
[recursive] := '|' [cmd] [recursive]; or
:= '|' [cmd] [terminate].
[terminate] := ""; or
:= '>‘ [filename]; or
:= ‘>>’ [filename].
[cmd] := [cmdname] [arg]*
[cmdname] := A string without any space, tab, > (ASCII 62), < (ASCII 60
[arg] := A string without any space, tab, > (ASCII 62), < (ASCII 60), |
[filename] := A string without any space, tab, > (ASCII 62), < (ASCII 6
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
2023/3/2 01:09 Lab 2: Shell
Here are some examples of invalid commands:
cat << file.txt
cat < file.txt < file2.txt
cat < file.txt file2.txt
cat > file.txt > file2.txt
cat > file.txt >> file2.txt
cat > file.txt file2.txt
cat > file.txt | cat
cat | cat < file.txt
cd / > file.txt
If there is any error in parsing the command, then your shell should print the following error message to and prompt for the next command.
Note that there should be a newline at the end of the error message. For example:
(Again, the nal █ character represents your cursor.)
You can specify a program by either an absolute path, a relative path, or base name only.
Error: invalid command
[nyush lab2]$ cat <
Error: invalid command
[nyush lab2]$ █
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
smargorp gnitacoL
2023/3/2 01:09 Lab 2: Shell
1. An absolute path begins with a slash ( / ). If the user speci es an absolute path, then your shell must run the program at that location.
2. A relative path contains, but not begins with, a slash ( / ). If the user
speci es a relative path, then your shell should locate the program by following the path from the current working directory. For example,
is equivalent to .
3. Otherwise, if the user speci es only the base name without any slash
( / ), then your shell must search for the program under . For example, when the user types ls , then your shell should try
. If that fails, it is an error. In this case, your shell should not search the current working directory. For example, suppose there is a program named in the current working directory. Entering
should result in an error, whereas runs the program.
In any case, if the program cannot be located, your shell should print the following error message to and prompt for the next command.
After creating the processes, your shell must wait until all the processes have stopped running—either terminated or suspended. Then, your shell should prompt the user for the next command.
Your shell must not leave any zombies in the system when it is ready to read the next command from the user.
If a user presses or , they don’t expect to terminate or suspend the shell. Therefore, your shell should ignore the following signals:
, and . All other signals not listed here should keep the default signal handlers.
dir1/dir2/program
./dir1/dir2/program
/usr/bin/ls
Error: invalid program
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
noisnepsus dna noitanimret ssecorP
gnildnah langiS
2023/3/2 01:09 Lab 2: Shell
Note that only the shell itself, not the child processes created by the shell, should ignore these signals. For example,
Here, the signal generated by terminates only the process cat , not the shell itself.
As a side note, if your shell ever hangs and you would like to kill the shell, you can still send it the or signal. To do so, you can connect to the running Docker container from another terminal window using:
...and then kill your shell using:
Sometimes, a user would read the input to a program from a le rather than the keyboard, or send the output of a program to a le rather than the screen. Your shell should be able to redirect the standard input ( ) and the standard output ( ). For simplicity, you are not required to redirect the standard error ( STDERR ).
Input redirection is achieved by a < symbol followed by a lename. For example:
[nyush lab2]$ cat
[nyush lab2]$ █
2250]# killall nyush
docker exec -it 2250 bash
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
noitcerider tupnI
noitcerider O/I
2023/3/2 01:09 Lab 2: Shell
If the le does not exist, your shell should print the following error message to and prompt for the next command.
Output redirection is achieved by > or >> followed by a lename. For example:
If the le does not exist, a new le should be created. If the le already exists, redirecting with > should overwrite the le (after truncating it), whereas redirecting with >> should append to the existing le.
A pipe ( | ) connects the standard output of the rst program to the standard input of the second program. For example:
The user may invoke n programs chained through (n – 1) pipes. Each pipe connects the output of the program immediately before the pipe to the input of the program immediately after the pipe. For example:
Error: invalid file
[nyush lab2]$ ls -l > output.txt
[nyush lab2]$ ls -l >> output.txt
[nyush lab2]$ cat shell.c | wc -l
[nyush lab2]$ cat shell.c | grep main | less
[nyush lab2]$ cat < input.txt
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
noitcerider tuptuO
2023/3/2 01:09 Lab 2: Shell
Here, the output of is the input of grep main , and the output of is the input of less .
Every shell has a few built-in commands. When the user issues a command, the shell should rst check if it is a built-in command. If so, it should not be executed like other programs.
In this lab, you will implement four built-in commands: cd , jobs , fg , and exit .
This command changes the current working directory of the shell. It takes exactly one argument: the directory, which may be an absolute or relative path. For example:
If cd is called with 0 or 2+ arguments, your shell should print the following error message to and prompt for the next command.
If the directory does not exist, your shell should print the following error message to and prompt for the next command.
cat shell.c
[nyush lab2]$ cd /usr/local
[nyush local]$ cd bin
[nyush bin]$ █
Error: invalid command
Error: invalid directory
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
sdnammoc ni-tliuB
2023/3/2 01:09 Lab 2: Shell
This command prints a list of currently suspended jobs to STDOUT , one job per line. Each line has the following format: . For example:
[index] command
[nyush lab2]$ jobs
[1] ./hello
[2] /usr/bin/top -c
[3] cat > output.txt
[nyush lab2]$ █
(If there are no currently suspended jobs, this command should print nothing.)
A job is the whole command, including any arguments and I/O redirections. A job may be suspended by , the signal, or the
signal. This list is sorted by the time each job is suspended (oldest rst), and the index starts from 1.
For simplicity, we have the following assumptions:
There are no more than 100 suspended jobs at one time.
There are no pipes in any suspended jobs.
The only way to resume a suspended job is by using the fg command (see below). We will not try to resume or terminate a suspended job by other means. We will not try to press or while there are suspended jobs.
You don’t need to worry about “process groups.” (If you don’t know what process groups are, don’t worry.)
The jobs command takes no arguments. If it is called with any arguments, your shell should print the following error message to and prompt for the next command.
Error: invalid command
fg
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
2023/3/2 01:09 Lab 2: Shell
This command resumes a job in the foreground. It takes exactly one argument: the job index, which is the number inside the bracket printed by the jobs command. For example:
[nyush lab2]$ jobs
[1] ./hello
[2] /usr/bin/top -c
[3] cat > output.txt
[nyush lab2]$ fg 2
The last command would resume in the foreground. Note that the job index of would become 2 as a result. Should the job be suspended again, it would be inserted to the end of the job list:
cat > output.txt
/usr/bin/top -c
/usr/bin/top -c
[nyush lab2]$ jobs
[1] ./hello
[2] cat > output.txt
[3] /usr/bin/top -c
[nyush lab2]$ █
If fg is called with 0 or 2+ arguments, your shell should print the following error message to and prompt for the next command.
If the job does not exist in the list of currently suspended jobs, your shell should print the following error message to and prompt for the next command.
Error: invalid command
Error: invalid job
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
2023/3/2 01:09 Lab 2: Shell
This command terminates your shell. However, if there are currently suspended jobs, your shell should not terminate. Instead, it should print the following error message to and prompt for the next command.
The exit command takes no arguments. If it is called with any arguments, your shell should print the following error message to and prompt for the next command.
Error: there are suspended jobs
Note that if the of your shell is closed (e.g., by pressing prompt), your shell should terminate regardless of whether there are suspended jobs.
We will grade your submission in an x86_64 Rocky Linux 8 container on Gradescope. We will compile your program using gcc 12.1.1 with the C17 standard and GNU extensions.
You must provide a , and by running make , it should generate an executable le named in the current working directory. (Refer to Lab 1 for an example of the Makefile .)
Your program must not call the function or execute .
Otherwise, what is the whole point of this lab?
Beat up your own code extensively. Better yet, eat your own dog food. I would happily use as my main shell (at least for the duration of this lab), so why wouldn’t you?
Error: invalid command
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
noitaulavE
noitalipmoC
2023/3/2 01:09 Lab 2: Shell
We are providing a sample autograder with a few test cases. Please extract them inside the Docker container and follow the instructions in the
le. (Refer to Lab 1 for how to extract a le.)
Note that these test cases are not exhaustive. The test cases on Gradescope are different from the ones provided and will not be disclosed. Do not try to hack or exploit the autograder.
You must submit a .zip archive containing all les needed to compile in the root of the archive. You can create the archive le with the
following command in the Docker container:
Note that other le formats (e.g., rar ) will not be accepted.
You need to upload the .zip archive to Gradescope. If you need to acknowledge any in uences per our academic integrity policy, write them as comments in your source code.
The total of this lab is 100 points, mapped to 15% of your nal grade of this course.
Compile successfully and can print the correct prompt. (40 points) Process creation and termination. (20 points)
Simple built-in commands ( cd and exit ) and error handling. (10 points)
Input and output redirection. (10 points)
Pipes. (10 points)
Handling suspended jobs ( jobs and fg ). (10 points)
$ zip nyush.zip Makefile *.h *.c
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
noissimbuS
Programming Help
2023/3/2 01:09 Lab 2: Shell
You will get 0 points for this lab if you call the function or execute /bin/sh .
Please make sure that your shell prompt and all error messages are exactly as speci ed in this document. Any discrepancy may lead to point deductions.
This lab is signi cantly more challenging than Lab 1 and requires signi cant programming effort. Therefore, start as early as possible! Don’t wait until the last week.
Please review our academic integrity policy carefully before you start.
This lab requires you to write a complete system from scratch, so it may be daunting at rst. Remember to get the basic functionality working rst, and build up your shell step-by-step.
Here is how I would tackle this lab:
Milestone 1. Write a simple program that prints the prompt and ushes
. You may use the system call to get the current working
directory.
Milestone 2. Write a loop that repeatedly prints the prompt and gets the user input. You may use the library function to obtain user input.
Milestone 3. Extend Milestone 2 to be able to run a simple program, such as ls . At this point, what you’ve written is already a shell! Conceptually, it
might look like:
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
!!!etanitsarcorp t’noD !!etanitsarcorp t’noD !etanitsarcorp t’noD
?detrats teg ot woH
Code Help
2023/3/2 01:09 Lab 2: Shell
Milestone 4. Extend Milestone 3 to run a program with arguments, such as .
Milestone 5. Handle simple built-in commands ( cd and exit ). You may use the system call to change the working directory.
Milestone 6. Handle output redirection, such as Milestone 7. Handle input redirection, such as
Milestone 8. Run two programs with one pipe, such as . The example code in is very helpful.
Milestone 9. Handle multiple pipes, such as
Milestone 10. Handle suspended jobs and related built-in commands
( jobs and fg ). Read to see how to wait for a stopped child. Some students may nd handling suspended jobs easier than implementing
pipes, so feel free to rearrange these milestones as you see t.
Keep versions of your code! Use git or similar tools, but don’t make your
repository public.
Your shell will make many system calls. Here are a few that you may nd
Process management: , (or other variants), waitpid() .
I/O redirection and pipes: , , .
Signal handling: signal() .
cat > output.txt
cat < input.txt
man 2 pipe
cat | cat | cat
man waitpid
access() ,
Click to reveal spoiler
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
sllac metsys lufesU
2023/3/2 01:09 Lab 2: Shell
Built-in commands: .
You might not need to use all of them, and you are free to use other system
calls not mentioned above.
Check the return values (and possibly, ) of all system calls and library function calls from the very beginning of your work! This will often catch errors early, and it is a good programming practice.
Man pages are of vital importance for programmers working on Linux and such. It’s a treasure trove of information.
Man pages are divided into sections. Please see for the description of each section. In particular, Section 2 contains system calls. You will need to look them up a lot in this lab.
Sometimes, you need to specify the section number explicitly. For example, shows the kill command in Section 1 by default. If you need to
look up the system call, you need to invoke .
You might nd writing the command parser troublesome. Don’t be frustrated. You are not alone. However, it is an essential skill for any programmer, and it often appears in software engineer interviews. Once you get through it, you will never be afraid of it again.
I personally nd the function extremely helpful. You don’t have to use it, but why not give it a try?
By default, when a program forks, gdb will continue to debug the parent process, and the child process will run unimpeded.
strtok_r()
man 2 kill
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
? retfa ssecorp dlihc eht gubed ot woh , nI
enil dnammoc eht gnisraP
segap nam gnidaeR
2023/3/2 01:09 Lab 2: Shell
If you want to follow the child process instead of the parent process, use the command .
If you want to debug both the parent and child processes, use the command . Then, you can use to show all
processes and use to switch to another process.
If you can already support I/O redirection for a single process, adding support for pipes shouldn’t be too dif cult since a pipe is just two I/O redirections. Start by reading the example code in and writing a simple program to see how pipe works.
Here are some additional hints:
Click to reveal spoiler
Click to reveal spoiler
This lab has borrowed some ideas from Prof. Arpaci-Dusseau and Dr. T. Y. Wong.
set follow-fork-mode child
set detach-on-fork off
info inferiors
man 3 pipe
https://cs.nyu.edu/courses/spring23/CSCI-GA.2250-002/nyush#builtincommands
?stnih erom ynA .sboj dednepsus tuoba desufnoc llits m’I
?stnih erom ynA .sepip tuoba desufnoc llits m’I