Linux basics for web developers

Introduction

Every web developer at some point finds himself in the position of hosting his application on a VPS that is based on the Linux operating system and needs basic knowledge of Linux server administration.
To facilitate the administration of linux servers, specialized applications such as “Control panel”, “Plesk” are often used… However, these applications are not free, what’s more, they often cost almost the same as renting the VPS itself. Even if a server administration license is leased, it is considered desirable (necessary) for a web developer to have basic knowledge of working with the Linux operating system. I have tried in this article to record data related to the linux operating system and its basic use through the bash shell (terminal).

Bash shell

For work with the Linux operating system, Bash Shell (terminal) is provided, which allows the user to:

• executing commands from the command line
• history of previously executed commands
• job control (job control)
• use of shell functions and aliases
• arithmetic operations

NOTE: On windows 10 there is a possibility to activate the Linux subsystem with all linux options, with which we simply switch windows CMD to “bash”. You will see more about this in the article “Installation tools for web development”.

Bash script

A file that contains a series of sequentially listed commands that are executed in the order in which they are listed is called a bash script and has the extension .sh.

It should be noted that the Unix system represents everything within the operating system as sequences of bytes without a structure, this includes files as well as devices connected to a computer with a Linux operating system. Therefore, all hardware (network cards, hard drives, partitions, keyboard, printer) is treated as a file-like object. Each hardware component is represented by a special file that has an appropriate name and place in the file system (eg system memory is defined as a file with the location /dev/mem, and the terminal (keyboard and screen) is defined in the file /dev/tty1).

“In Linux everything is a file”

Access to server using SSH protocol

What is the SSH protocol?

The SSH protocol is most often used for connection and communication with a remote Linux system. Secure Shell (SSH) is a cryptographic network protocol for the secure operation of network services over an unsecured network. SSH commands are encrypted and secure because both ends of the connection, both client and server, are authenticated using a digital certificate, and passwords are encrypted.
The protocol works according to the client-server model, which means that the connection is initiated by the SSH client connecting to the SSH server. The SSH client handles the connection establishment process and uses public key cryptography to verify the identity of the SSH server. After the setup phase, the SSH protocol uses strong symmetric encryption algorithms to ensure the privacy and integrity of the data exchanged between the client and the server.

Connection to the system with SSH protocol

Forconnection to the remote server is required:

1. hostname IP
2. username of the host
3. password for the user

The remote server is accessed using the appropriate commands, depending on whether your user is the same as on the server or we don’t have two different commands. If your user is the same as on the server we can use:

But if it is not the same user then we use:

After this, the system requires the appropriate code to be entered, finally the command is used to terminate the session:

exit

Applications that support the SSH protocol

To access the remote server, we need bash (terminal), there is the already mentioned option to switch Windows’ CMD to bash, but this only applies to WIN 10, for users of older Windows editions, the solution is one of the third-party applications that emulate linux bash:

• Putty
  Putty is the oldest Windows application for communicating with a remote system that uses the SSh protocol, while most other applications are “polished” variations with additional features.
• PuttyTray
• Kitty
• Mobaxterm
• mRemoteNG
• Smartty

Copying and synchronizing files with Rsync

Rsync is a fast and highly customizable file copying tool. It can copy files locally as well as from another host. It offers a large number of options and allows flexible specification of the selection of files to be copied. It is known for its “delta” transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and existing files in the destination.

There are two different ways for Rsync to contact a remote system: using SSH or directly contacting dynamic Rsync over TCP.

NOTE:
Using the SSH protocol, data is transferred over a secure connection with encryption. To use the SSH protocol, you must specify the root password to connect, and using the SSH option will encrypt the encrypted mode so that your password will be secure.

Additional options are:

• -v => the verbose option simply produces screen output that provides comments on the work as it occurs, so you’ll be displaying the real-time status of the user program or the task launcher or the commands you’ve submitted.
• -r => copies data recursively (copying a directory and all its files and subdirectories and all their files, deeper and deeper to the bottom of the directory tree).
• -a => archive mode enables recursive copying of files while preserving symbolic links, file permissions, ownership and timestamps.
• -z => compress files
• -h => human-readable format
• -d => directory transfer without recursion
• -e => specify the ssh as remote shell

a) Work with local files

b) Synchronization from local to remote server

General appearance of the command:

Example

c) Synchronization from remote server to local

Generalcommand layout:

Example

Linux file system structure

The picture shows the most common file system structure of the Linux operating system. The most important directory is Root and it is the only one without a parent directory. It is marked with a slash “/” and only the “root” user has access to this directory. If the path that defines the location of a file or directory begins with a slash “/”, we know that it is an absolute path. Relative path is defined relative to the current position and starts with “./”, as this is how the current directory is marked.

/bin

This directory stores binary executable commands, such as: ps, ls, ping, grep, cp…

/etc

Contains configuration files for all programs, as well as shell scripts for starting and shutting down applications.

/dev

Linux represents devices as files, so this directory contains several special files that represent devices. For example “/dev/sda” represents the first SATA drive in the system.

/home

This folder stores private and configuration files for each user. Each user has only “write” access in his directory, to write in other directories he needs to be the root user.

/media

This directory contains subdirectories that display removable media inserted into the computer.

/root

This directory is the directory of the root user. Instead of being at “/home/root”,with other users he is here.

/srv

This directory contains data for services provided by the system. For example if you use Apache HTTP Server to serve web pages, you will probably store your website files in a directory inside the /srv directory.

/tmp

Applications store temporary files in this directory. The files in this directory are generally deleted whenever your system reboots and can be deleted at any time by utilities such as “tmpvatch”.

/usr

This directory contains applications and files used by users, as opposed to applications and files used by the system. For example non-essential applications are located inside the “/usr/bin” directory instead of the /bin directory, and the local “/usr/local” directory is used as a place to install local applications, preventing them from unnecessarily “polluting” the rest of the system.

/var

Changeable (variable) files are recorded in this directory. Files whose contents are expected to grow can be found under this directory. This usually includes: system log files (/var/log), packages and database files (/var/lib), or emails (/var/mail)…

Basic linux commands

Help writing commands

When working with the Bash shell, we use:

• Tab on the keyboard is a huge help when typing Linux commands, as it will automatically fill in file and directory names.
• The symbols “?” and “*” are wildcard in Linux. The question mark “?” will replace a single character, while “*” will replace the entire string.
• Text is copied in the terminal with the Ctrl+Shift+C command, while it is pasted with the Ctrl+Shift+V command (if the command does not work, try right-clicking or the command from the context menu).

Command syntax

All commands in Linux can be represented with the following pattern:

Bash Shortcuts

Monitoring and information

Working with the file system

Command Explanation pwd

Prints the current directory path to the terminal

mkdir test-direktorijum

Creates a new directory test-directory

cd test-direktorijum

Entering directory test-directory

cd ..

Go up a directory

ls

Lists all files in the current working directory

ls -a

Lists all files in the current working directory including hidden

ls -R

Recursively lists all files in the current working directory including all subdirectories

ls -t

Lists all files in the current working directory sorted by the filter last modified

ls -S

Lists all files in the current working directory sorted by the filter size

ls -l

Lists all files in the current working directory in full format

ls -1

Lists all files in the current working directory – each file on a new line

ls -m

Lists all files in the current working directory, files are separated by commas.

touch novi_fajl.txt

Creates a new file named new_file.txt

less neki_fajl

Print the contents of the text file some_file in the terminal (exit with “Q”)

cat neki_file1 neki_file2

Prints the textual content of the files in the terminal (if there are more, merge them all into one printout)

file file1

Prints information about the file file1

in the terminal cp pic.jpg pic2.jpg

Copies the file in the working directory pic.jpg and saves it as a new file named pic2.jpg also in the working directory cp /home/pic.jpg /home/backup/pic.jpg

The file pic.jpg is copied from the home directory and saved in the backup directory as a new file named pic.jpg

cp -R /home/slike /home/backup

Copies the entire image directory with all subdirectories to the backup folder (the first one becomes a subdirectory of the backup folder)

cp *.jpg /home

Copy all files with the extension .jpg in the working directory and save them in the home directory with the same names

mv /home/file1 /home/file2

Rename file file1 from folder home to file2

mv /home/file1 /tmp/file2

Moves file file1 from folder home to folder tmp and saves it as file file2

rm file1

Deletes the file file1

head file1

Prints the first 10 lines of the text file file1

to the terminal tail file1

Prints to the terminal the last 10 lines of the text file file1

tail -3 file1

Prints in the terminal the last 3 lines of the text file file1

Search

Command Explanation grep "neki tekst" /home/logovi.txt

Search for the text “some text” within the file logovi.txt. If there is a result, it is printed in the terminal

grep -r "neki tekst" /home/

Recursive text search through all files within the home directory and its subdirectories

grep -i "neki tekst" /home/

Case insensitive text search “some text” through all files within the home directory

find /home/ -name "test*"

Search for files in the home directory by name (-name), which starts with test

find /home/ -user "pera"

Search for files in the home directory by user (-user), whose name starts with pen

find /home/ -mmin 60

Search for all files in the home directory that have been modified in the last 60 minutes

find /home/ -amin 60

Search for all files within the home directory that have been accessed in the last 60 minutes

find /home/ -size 60M

Search for all files in the home directory that are larger than 60MB

find /home/ -size +50M -size -100M

Search for all files in the home directory that are larger than 50MB and smaller than 0d 100MB

find /home -type f -empty

Search for all files in the home directory that are empty

find /home -type d "test"

Search for directories within the home directory named “test”

find /home -type d -empty

Search for directories within the home directory that are empty

File permissions

Command Explanation chmod 644 file.html

Owner can read and write (4+2), group can read, others can also read chmod u=rw example.jpg

Owner can read and write (4+2) chmod -R 755 nekifolder

Recursively change the permissions on the folder named “nekifolder” and all its subfolders so that owner can read and write and execute (4+2+1), group can read and execute (4+0+1), and others can read and execute (4+0+1) chmod 775 file

Change of permissions Change mode of file to 775

chmod -R 600 nekifolder

Recursive change of permissions to 600 for somefolder and all its files and subfolders

chown user:group nekifile

Change ownership of file nekifile not group

EXPLANATION:

User: Read, Write, and eXecute => u=rwx
Group: Read and eXecute => g=rx
Others: Read => o=r

• “row” = 4
• “write” = 2
• “execute” = 1
• “no permission = 0
• 5 is a combination of permissions 4+0+1 (read, no write, and execute)
• 6 is a combination of permissions 4+2+0 (read, write, and no execute)
• 7 is a combination of permissions 4+2+1 (read, write, and execute)

chmod u=rwx,g=rx,o=r nazivfajla is equivalent to chmod 754 nazivfajla

Archiving

The command “tar” merges several files into one, i.e. creates an archive, such an archive can be compressed with the “gzip” algorithm.

Command Explanation tar -cf archive.tar file1 file2 Creating one archive from two files tar -cf archive.tar mydir/ Creating archive from directory “mydir” tar -czvf naziv_arhive.tar.gz /home/test

From the selected directory “test” creates a compressed archive named archive_name.tar.gz

tar -czvf naziv_arhive.tar.gz test In the working directory from the selected directory “test” creates a compressed archive named archive_name.tar.gz tar -xf archive.tar Extracts archive tar -xzvf archive.tar.gz Extracts a compressed archive tar -xvf archive.tar documents/work/budget.doc Extracts only the documents/work/budget.doc file from archive.tar tar -xvf archive.tar documents/work/ Extracts only the documents/work/ directory and its files from the archive.tar archive tar -czf DogPhotos.tar.gz --exclude='kitty.jpg' MyPetPhotos Creates an archive DogPhotos.tar.gz from the files in the directory “MyPetPhotos” from which the file named kitty.jpg has been extracted

EXPLANATION:
–create or (-c) => creates a new .tar archive
–verbose or (-v) – the verbose option displays the status of the user program
–file or (-f) => indicates that the next argument in the command will be the name of the archive file
–gzip or (-z) => compress archive with gzip algorithm
–extract or (x) – extracts archive
–ignore-case => ignore case when searching against the given pattern
–overwrite => deletes the initial uncompressed file and replaces it with a compressed
–recursion => recursion is the default
setting
–no-recursion => no recursion
–exclude=PATTERN => excludes the defined pattern from the selection

Pipe

Pipe is a form of redirection used in Linux and other Unix-like operating systems to send the output of one program to another program for further processing. It is marked with a vertical line | between two commands.

komanda_1 | komanda_2 | komanda_3

In the previous example, command_1 will be executed first, after which command_2 will take its output as input parameters and process them, and finally command_3 will take the processed data and also process them.

Example

ls -al | grep '^d'

This command combination lists only the subdirectories of the current working directory.

Example

$ls -l | grep -i "neki_naziv"

This combination of commands lists the contents of the current working directory, and then filters by the desired name (case-insensitive -i = insenitive).

Example

ls -la | sort -nk5

This combination of commands lists the contents of the current working directory, and sorts it according to the numerical values in the 5th column.

Cron job

Cron is the name of a program that allows unix users to execute commands or scripts (groups of commands) automatically at a specific time/date. Cron is a daemon, which means it only needs to be started once, after which it will wait for a command. The Cron daemon uses cron tables to define which tasks it should run and when. Each line of the cron table represents one job to be executed in a defined time period.

Cron syntax

The format of one line of the cron table looks like this:

Crontab

Crontab is a command used to list the tables used by the cron daemon.