UNIX Commands

About UNIX

UNIX stands for UNiplexed Information Computing System (UNICS), also known as UNIX. A UNIX operating system is a multitasking operating system that allows you to initiate more than one task from the same terminal.

LINUX is an advanced version of UNIX. It has several features similar to UNIX, still has some key differences. Linux is open-source software and can be used freely without any licensing fees.

Why UNIX is popular than Windows?

UNIX-based systems are inherently more secure than the Windows operating system.

Shell in UNIX

A shell is an environment in which we can run our Unix commands, programs, and shell scripts.

Examples:

Bourne shell (sh)
Korn shell (ksh)
Bourne Again shell (bash)
C shell (csh)

Tools used for UNIX in real-time

To work with UNIX we have two free tools available: PuTTY and WinSCP.

PuTTY — a UNIX CLI (Command Line Interface) tool used to run UNIX commands.
WinSCP — a UNIX GUI (Graphical User Interface) tool used to access files on the UNIX operating system.

Why should I learn UNIX as an Informatica Developer?

We have a Command Task in Informatica — UNIX / Windows commands are used inside the Command Task.
In real-time, the Informatica Server is installed on UNIX. The INFA_SHARED folder lives on the UNIX server. To access INFA_SHARED and source / target / other files you need UNIX commands.
We can schedule Informatica jobs in UNIX using crontab.

UNIX Commands Reference

1. pwd

Print the Present Working Directory in UNIX.

bash

$ pwd

2. mkdir

Create a directory in UNIX.

bash

mkdir dir1

Create more than one nested directory at once:

bash

mkdir dir2
mkdir dir2/dir3
mkdir dir2/dir3/dir4

3. rmdir

Remove a directory in UNIX.

4. cd

Change directory in UNIX.

bash

cd dir2
cd ..       # come back from the directory

5. cal

Display the calendar.

6. date

Display the system date and time.

Syntax: date [+format]

Example — display the date in dd/mm/yy format:

bash

date +%d/%m/%y

7. Creating a file in UNIX

a) Using cat command

bash

cat > abc.txt     # creates a file; press Ctrl-D to exit
cat filename      # display the contents of a file
cat >> filename   # append to a file

b) Using touch command

bash

touch file2.txt                          # create an empty file
touch file1 file2 file3                  # create multiple empty files at once
cat filename                             # display content of a file
cat -b filename                          # display content with line numbers

8. ls — listing directories and files

All data in UNIX is organised into files. All files are organised into directories. These directories are organised into a tree-like structure called the filesystem.

Use the ls command to list out all the files or directories available in a directory.

ls -l — list with long format (show permissions)
ls -a — list all files including hidden files starting with .
ls -R — recursive directory tree list
ls -t — sort by time & date
ls -r — list in reverse order
ls -ls — list with long format with file size
ls -lrt — long format, reverse, sorted by time

Wildcards: * matches 0 or more characters; ? matches a single character.

bash

ls file*     # files whose names start with "file"
ls *.txt     # files ending with ".txt"

9. whoami

Displays the user id of the currently logged-in user.

10. who

Displays the list of users currently logged in.

11. wc — counting words in a file

bash

wc file_name

You can pass multiple files at once:

bash

wc filename1 filename2 filename3

wc -l state.txt — number of lines in a file
wc -l state.txt capital.txt — works across multiple files
wc -w state.txt — number of words in a file
wc -c state.txt — count of bytes in a file
wc -m state.txt — count of characters in a file
wc -L demo_file — length of the longest line in the file

13. cp — copying files

Syntax: cp source_file destination_file

bash

cp filename1 filename2   # copy contents of filename1 to filename2

14. mv — renaming files

Syntax: mv old_file new_file

bash

mv filename newfile   # rename "filename" to "newfile"

15. rm — deleting files

bash

rm filename                       # remove a file completely
rm filename1 filename2 filename3  # remove multiple files at once

16. chmod — change file/directory permissions

4 stands for read
2 stands for write
1 stands for execute
0 stands for no permission

Set read + write for other users:

bash

chmod o+w *.txt

Remove the write permission for other users:

bash

chmod o-w *.txt

Set execute permission for all users:

bash

chmod a+x File1
chmod 777 testfile

17. Sending email

Syntax: mail [-s subject] [-c cc-addr] [-b bcc-addr] to-addr

Send a test message:

bash

mail -s "Test Message" admin@yahoo.com

Piping commands

You can connect two commands together so that the output from one program becomes the input of the next. Two or more commands connected this way form a pipe (|).

19. ps

Display currently running processes.

20. kill

Kill the current process.

21. man — manual / help

Interface for working with the online reference manuals.

Syntax: man [-s section] item

bash

$ man cat   # show manual page for the cat command

22. find — search for files and directories

Syntax: find [starting-point] [expression]

bash

$ find
# list all files found in the current directory and its hierarchy

$ find . -name cust.dat
# find all files named cust.dat from all directories starting at .

$ find /root/infa_shared/SrcFiles -name File1.dat
# find files named File1.dat under /root/infa_shared/SrcFiles

$ find /root/infa_shared/SrcFiles -iname File1.dat
# case-insensitive variant (matches File1.dat AND file1.dat)

find / -type d -name dir10
# find all directories named dir10 anywhere on the filesystem

find . -type f -empty
# find all empty files in all directories

23. du — disk usage

Estimate disk usage in blocks.

Syntax: du [options] [file]

bash

$ du                # blocks occupied by files in the current directory
du -sh *            # summary of directories (-s) in human-readable format
du -sk *            # summary of directories (-s) in kilobytes (-k)

24. df — disk free

Show number of free blocks for mounted file systems.

Syntax: df [options] [file]

bash

$ df -l   # show number of free blocks in local file systems

UNIX filter commands

grep · sort · uniq · cat · more · cut · paste · head · tail · wc · tr — these are the filter commands you reach for most often when shaping streams of text.

25. grep — search for a pattern

Searches a file or files for lines that have a certain pattern. Comes from g/re/p — globally search for a regular expression and print all lines containing it.

bash

grep "this" demo_file        # search for the string in a single file
grep "this" demo_*           # check the string in multiple files
grep -i "this" demo_file     # case-insensitive search
grep -r "this" *             # recursive — current dir and all subdirs
grep -l this demo_*          # display only matching file names
grep -n "this" demo_file     # show line number with each match
grep "[a-e]" file1           # lines containing any of a, b, c, d, e
grep "[^aeiou]" file1        # lines with NO vowel
grep "^hello" file1          # lines that start with "hello"
grep "done$" file1           # lines that end with "done"
grep -e "Agarwal" -e "Aggarwal" -e "Agrawal" geekfile.txt

^ — exclude ([^aeiou]) OR anchor to the start of a line
$ — anchor to the end of a line

26. sort — order lines

Arrange lines of text alphabetically or numerically.

Initial contents of file1.txt:

text

01 Priya
04 Shreya
03 Tuhina
02 Tushar

bash

$ sort file1.txt

text

01 Priya
02 Tushar
03 Tuhina
04 Shreya

bash

$ sort -r file1.txt   # reverse order

bash

$ sort -k 2 file1.txt # sort by the second field

27. uniq — report / filter repeated lines

bash

uniq kt.txt              # display unique lines
uniq -c kt.txt           # number of times a line was repeated (count)
uniq -d kt.txt            # only repeated lines (-d duplicate)
uniq -u kt.txt            # only show lines that are NOT repeated (unique)

Delete duplicate lines (interview favourite)

``sort file.txt | uniq -u`

 — remove duplicate lines from a file in UNIX.

`sort file.txt | uniq -u | cat file.txt`` — remove and display the contents of the file.

28. head

head by default prints the first 10 lines of each file to standard output.

bash

head file1.txt   # display first 10 lines of file1.txt

29. tail

tail by default prints the last 10 lines of each file.

bash

tail file1.txt   # display last 10 lines of file1.txt

Display the *n*th line of a file:

bash

head -10 filename | tail -1 filename   # 10th line of the file
head -n filename | tail -1             # nth line of the file

30. diff — compare two or more files

bash

diff a.txt b.txt

31. tr — translate / delete characters

Syntax: tr [OPTION] SET1 [SET2]

bash

$ cat greekfile | tr "[a-z]" "[A-Z]"   # lower → UPPER
$ cat greekfile | tr "[A-Z]" "[a-z]"   # UPPER → lower

32. cut — extract sections from each line

Syntax: cut OPTION… [FILE]…

bash

cut -c 2,5,7 state.txt   # print 2nd, 5th and 7th characters of each line

33. zip / unzip

bash

zip archivename.zip filename1 filename2 filename3
unzip archivename.zip

34. echo

Display a line of text / string that is passed as an argument.

bash

echo Hello world

35. rev — reverse characters per line

bash

rev file1.txt   # reverse the contents of the file

Reverse a string:

bash

echo "java" | rev   # → avaj

36. sed — stream editor

A lot of functions on a file: searching, find and replace, insertion or deletion.

Syntax: sed OPTIONS… [SCRIPT] [INPUTFILE…]

bash

sed 's/unix/linux/' geekfile.txt   # replace "unix" with "linux" in the file

37. awk — scripting for reports

awk is a scripting language used for manipulating data and generating reports. It requires no compiling and allows variables, numeric and string functions, and logical operators.

awk operations:

Scans a file line by line
Splits each input line into fields
Compares input line / fields to a pattern
Performs action(s) on matched lines

Useful for:

Transform data files
Produce formatted reports

Programming constructs:

Format output lines
Arithmetic and string operations
Conditionals and loops

bash

awk '{print}' employee.txt              # print every line
awk '/manager/ {print}' employee.txt    # print only lines matching 'manager'

38. history

Give the entire history of the commands you have used.

bash

history > a.txt