How to increase available memory on any Linux server

If you're suffering with server lag, ping latency, degraded system functionality or generally slow response times you may need to add extra RAM to your server.

Of course, adding RAM is a solution - but it's not the only option you have. Why not think about utilising swap space, this enables you to let your HDD act as a virtual-backup solution to out-of-memory situations.

These days of course using your HDD for RAM doesn't seem as bad as what it sounds, with SSD's widely used the temptation to switch to swap space is quite understandable.

Plus, for most use-cases, you will only need as much as 2-4GB of RAM in order to compile your favourite popular software title - such as Apache, or similar. To simply run the program may require less RAM thus less overhead for cost. You could use the package manager to install Apache to avoid having to compile it but why do that when you a) want to compile and b) can simply add swap space.

To add swap space, follow these instructions:

sudo fallocate -l 1G /swapfile

Here, we are creating a swap file with a size of 1G. If you need more swap, replace 1G with the desired size.

A good rule of thumb is to have 3x more swap than your physical RAM. So for 2GB of RAM you would have 6GB of swap space.

If the fallocate utility is not available on your system or you get an error message saying fallocate failed: Operation not supported, use the dd command to create the swap file:

sudo dd if=/dev/zero of=/swapfile bs=1024 count=1048576

Next, set the swap space files permissions so that only the root user can read and write the swap file:

sudo chmod 600 /swapfile

Next, set up a Linux swap area on the file:

sudo mkswap /swapfile

Activate the swap by executing the following command:

sudo swapon /swapfile

And that should do it. Check the memory is allocated by simply using free.

sudo free -h

Finally, make the changes permanent by adding a swap entry in the /etc/fstab file:

echo  '/swapfile swap swap defaults 0 0' >> /etc/fstab

Now, when your server reboots the swap space will be reconfigured automatically.


And that's it! If you enjoyed this post please feel free to leave a like or a comment using the messaging options below.

How to identify and locate large files and folders

When it happens that your VPS is eating data by the second and there is disk read/write issues one port of call you are bound to visit is searching and identifying large files on your system.

Now, you would have been forgiven for thinking this is a complicated procedure considering some Linux Bash solutions for fairly simple things, but no. Linux Bash wins again!

du -sh /path/to/folder/* | sort -rh

Here, du is getting the sizes and sort is organising them, -h is telling du to display human-readable format.

The output should be something like this:

2.3T    /path/to/directory
1.8T    /path/to/other

It does take a while to organise as it is being done recursively however given 3-5mins and most scenarios will be fine.

A memorable way to Find and Replace recursively

So yeah, getting used to Bash is about finding the right way to do things. However, learning one-liners and picking up information here and there is all very useful, finding something you don't need to Google in order to recall is extremely important.

Take the case of recursive find and replace. We've all been there, it needs to be done frequently but you're either a) scared or b) forgetful. Then you use the same snippet from a web resource again and again and eventually make a serious error and boom, simplicity is now lost on you!

So here it is, something you can remember so that you don't use different methods depending on what Google throws up today.

grep -Rl newertext . | xargs sed -i 's/newertext/newesttext/g'

Notice, we use grep to search -R recursively with results fed to xargs which does a simple sed replace with g global/infinite properties.

Say you want to keep it simple though and find and review the files before doing the "simple" replace. Well, do this.

grep -R -l "foo" ./*

The uppercase -R denotes to follow symlinks and the ./* indicates the current directory. The -l requests a list of filenames matched. Neat.

Get To Know Linux Bash In Under 30 Minutes

File Management Creation

mktemp

Creates a temporary file with a random name. Guarantees that the new file doesn’t exist

touch new_file.txt new_file_2.txt

Creates both new_file.txt and new_file_2.txt

touch file_name_{a..c}

Creates file_name_a, file_name_b, and file_name_c

Moving

cp file_name.txt file_name_copy.txt 

Creates a copy of file_name.txt named file_name_copy.txt in the same working directory

Reading

head file_name.txt 

Prints first 10 lines of a file

tail file_name.txt 

Prints last 10 lines of a file

touch new_file.txt 

Creates a new file

touch {new_file, new_file_2}.txt

Creates both new_file.txt and new_file_2.txt

touch file_name_{1..3}

Creates file_name_1, file_name_2, and file_name_3

`mv file_name.txt new_file_copy.txt`

Moves file_name.txt to new_file_copy.txt in the same directory, renaming the file

cat file_name.txt 

Prints full contents of a file

less file_name.txt

Prints a part of file contents

Deletion

rm file_name.txt 

Removes a file

wc file_name.txt

Prints number of lines words and characters in the file

rm -f file_name.txt

Removes a file ignoring non-existent files

Compression and Decompression

zip archive_name.zip file_name_1.php file_name_2.php
tar -cvf archive_name.tar file_name_1.php file_name_2.php 
tar -zcf archive_name.tar.gz file_name_1.php file_name_2.php

Archives separate files (depending on the format)

zip -r archive_name.zip directory
tar -cvf archive_name.tar directory 
tar -zcf archive_name.tar.gz directory

Archives a whole directory (depending on the format)

Replacing in Files

unzip archive_name.zip
tar -xvf archive_name.tar
tar -zxvf archive_name.tar.gz

Extracts an archive (depending on the format)

sed -i 's/search_query/replace_query/' file_name.txt

Modifies the content of the original file if the replacement query exists in the file

sed 's/search_query/replace_query/g' `file_name.txt` > `new_file_name.txt`

Replaces search_query with replace_query in file_name.txt and saves everything in new_file_name.txt

sed 's/search_query/replace_query/g' file_name.txt Replaces search_query with replace_query in file_name.txt

Search in Files

grep 'contents' /file_name.txt 

Searches for contents inside file_name.txt

grep 'contents_1|contents_2' /directory -R 

Searches for contents or contents_2 inside directory recursively

grep 'contents' /directory -i

Performs case-insensitive search

grep 'contents' /directory -x 

Matches the entire line and prints it out

grep 'contents' /directory -l

Only displays files that match contents query

Directories Navigation

cd directory

Goes to the listed sub-directory

cd -

Goes to the previous directory

ls -l

Shows where symbolic links are pointing

grep 'contents' /directory -r

Searches for contents inside directory recursively

grep 'contents' /directory -v 

Displays only the lines that don’t match contents

grep 'contents' /directory -i

Performs case-insensitive search

grep 'contents' /directory -n

Displays line numbers along with the results

grep 'contents' /directory -L

Only shows files that don’t match contents query

 cd

Goes to the home directory

ls

Lists all directories

ls -l -h

Lists all directories in long format, -h flag uses the human-readable format

cd ..

Goes the directory one level up from the current directory

ls -a

Lists all directories, including hidden ones

ls -t

List directories by modification time, showing newest first

cd ~
tree

Shows directory and file trees

stat `file.txt`

Lists file’s size alongside created and modified timestamps

Creation

mkdir new_directory 

Creates a new directory

mkdir -p parent/child/nested_child 

Creates nested directories

mktemp -d

Creates a temporary directory with a random name. Guarantees that the new directory doesn’t exist

Moving

mv old_directory new_directory Moves old_directory to new_directory
cp -r directory directory_copy

Copies directory to directory_copy recursively

tree -d

Shows directory tree tree -a Shows directory and file trees, including hidden ones

stat directory

Lists directory’s size alongside created and modified timestamps

pwd

Shows current directory path

  mkdir new_directory_1 directory_2 

Creates multiple new directories

mkdir -p {dir_one, dir_two}/nested 

Creates multiple nested directories

  cp directory directory_copy Copies directory to directory_copy

Deletion

rmdir directory 

Removes directory

rm -r directory

Removes dicrectory recursively

Symbolic Links

ln -s path link

Creates a symbolic link called link to the path directory

ln -s -f path link

Overwrites existing symbolic link called link

Popular Directory Permission

777 All possible permissions. Not recommended for security reasons.

755 Directory owner has full permissions. Other users can list the directory but won’t be able to manage the files.

700 Directory owner has full permissions. Other users have no permissions.

Permission Management

ls -l

Displays directory contents in a long format, which displays both permissions and ownership

chown :new_group file_name 

Changes group for file_name

chmod u=rwx, g=r, o-rwx file_name

Sets file_name read, write, execute permissions for the owner and other users, leaving read-only permissions for the group

chmod u+x file_name.txt

Sets the user permissions to execute

chmod u+x, g+x, o+x file_name.txt
chmod a+x file_name.txt
chmod +x file_name.txt

Sets everyone's permissions to execute

chown username:new_group file_name Changes both the owner and group

for file_name

 chown username file_name

Changes the owner for file_name

chmod 777 -R directory

Sets read, write and execute permissions for everyone in directory recursively

chmod g+x file_name.txt

Sets the group permissions to execute

Arrays Creation

indexed_array = (element_1, element_2, element_3) 

Creates an index array

declare -A associative_array = ([key_1] = element_1, [key_2] = element_2, [key_3] = element_3)

Creates an associative array

Adding Elements
indexed_array += (new_element) 

Adds a new element to an indexed array

Printing Out
echo ${indexed_array[0]} 

Prints out the first array element

associative_array += ([key_1] = new_element

Adds a new element to an associative array (note that the key needs to be provided as well)

echo ${indexed_array[@]} 

Prints out the whole array

  echo ${!associative_array[@]}

Prints out all the keys for an associative array

Deletion
unset indexed_array[2]

Removes the third element from an indexed array

unset associative_array[key]

Removes the key element from an associative array

Resource Usage and Processes

top htop

Lists out all the processes interactively

nice -n 10 process_name 

Changes the priority to 10 for process_name

kill 2468

Kills a process that has process ID 2468

 jobs -p

Shows all processes jobs alongside their IDs

 du

Shows current directory, sub-directories and file sizes

Shutdown and Reboot

ps all

Lists out all currently running processes

renice 10 2468

Changes the priority to 10 for a process that has process ID 2468

killall process_name

Kills all processes that have process_name in their name

lsof

Shows all open files and processes that use them

du /directory/sub-directory

Lists specified directory, sub-directories and file sizes

pidof process_name 

Prints out the process ID of process_name

ps -o ni 2468

Displays the priority for a process ID 2468

jobs

Shows all background processes

free

Displays memory usage

df

Shows disks alongside their used and available space

shutdown -r
reboot

Immediately reboots the system

shutdown

Shuts the system down after one minute

shutdown -r +10

Reboots the system after 10 minutes

shutdown now

Immediately shuts down the system

shutdown -c

Cancels a shutdown or reboot

shutdown +10

Shuts the system down after 10 minutes

reboot -f 

Forces a reboot

Scheduled Tasks Crontab Syntax

Access via crontab -e

[minute] [hour] [day] [month] [week] [command]

Each line represents one command to be run as directed

crontab -i

Will show a prompt before removing a user’s crontab

@daily cat /home/hello_world.sh

Schedules a background job to run every day

@reboot cat /home/hello_world.sh

Schedules a job to run after each system reboot

crontab -l

Used to view crontab entries (cron jobs), and display system crontab file contents

crontab -r

Will remove the current crontab file

* * * * * cat /home/hello_world.sh

Schedules a job to run every minute

00 08-17 * * * cat /home/hello_world.sh

Schedules a job to run every weekday, including weekends, from 8am to 5pm

@monthly cat /home/hello_world.sh

Schedules a job to run at the beginning of each month

0 12,15,17,19,21 * * * cat /home/hello_world.sh

Schedules a job to be run five times a day at 12pm, 3pm, 5pm, 7pm and, 9pm

HTTP Requests

curl https://domain.tld 

Returns response body for domain.tld

curl -o file.txt https://domain.tld Outputs to a text file

Network and DNS

ip addr

Shows all IP addresses on a system

ping -c 15 -i 3 domain.tld

Pings the domain 10 times, 3 seconds apart

traceroute domain.tld

Displays all servers the network traffic goes through

curl -i https://domain.tld

Returns response body for domain.tld and includes status code and HTTP headers

curl -H|--header "User-Agent: Agent" https://domain.tld 

Adds an HTTP header

ip route show

Shows all IP addresses to router

netstat -l

Shows all open ports

nmap 0.0.0.0

Scans for the 1,000 most commonly open ports on localhost

host example.net

Display IPv4 and IPv6 addresses for domain.tld

ping domain.tld

Sends multiple ICMP protocol ping requests netstat -i Shows all open ports with in/out usage

nmap 255.255.255.255

Scans for the 1,000 most commonly open ports on remote IP address

dig example.net

Display complete DNS information

dig example.net +short

Display complete DNS in short format

dig example.net ns 

Query NS records

dig example.net txt 

Query TXT records dig example.net A Query A records

dig example.net cname 

Query CNAME records

dig example.net MX Query MX records

Secure Shell Protocol (SSH)

ssh hostname

Connects to hostname using current username via default SSH port 22

ssh root@255.255.255.255 -p 1023

Connects via given username and IP via given SSH port

ssh root@255.255.255.255

Connects via given username and IP via default SSH port 22

What is Linux? An introduction to the most widely used Operating System on the planet

Linux is an open-source Operating System which is released with different flavours (or distros) under the guise of free-to-use software. Anybody can download and run Linux free-of-charge and with no restraints on the end-user; you could release, distribute and profit from Linux with relative ease with no worry of associated cost or licensing infringement.

It is fair to say Linux has formidably and profoundly revolutionised and defined the process of interacting with electronic devices. You can find Linux in cars, refrigerators, televisions and of course, as a desktop-grade or headless operating system. Once you become accustomed to Linux, you quickly see just why all the top 500 supercomputers all run Linux.

Linux has been around since the mid-1990’s and is is one of the most reliable, secure and hassle-free operating systems available. Put simply, Linux has become the largest open sources software project in the world. Professional and hobbyist programmers and developers from around the world contribute to the Linux kernel, adding features, finding and fixing bugs and security flaws, live patching and providing new ideas—all while sharing their contributions back to the community.

Wikipedia

Linux is a family of open-source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991, by Linus Torvalds.

Direct Link to Linux on Wikipedia

Open Source

Linux is a free, open source operating system, released under the GNU General Public License (GPL). Anyone can run, study, modify, and redistribute the source code, or even sell copies of their modified code, as long as they do so under the same license.

Command Line

The command line is your direct access to a computer. It's where you ask software to perform hardware actions that point-and-click graphical user interfaces (GUIs) simply can't ask.

Command lines are available on many operating systems—proprietary or open source. But it’s usually associated with Linux, because both command lines and open source software, together, give users unrestricted access to their computer.

Installing Linux

For many people, the idea of installing an operating system might seem like a very daunting task. Believe it or not, Linux offers one of the easiest installations of all operating systems. In fact, most versions of Linux offer what is called a Live distribution, which means you run the operating system from either a CD/DVD or USB flash drive without making any changes to your hard drive. You get the full functionality without having to commit to the installation. Once you’ve tried it out, and decided you wanted to use it, you simply double-click the “Install” icon and walk through the simple installation wizard.

Installing Software on Linux

Just as the operating system itself is easy to install, so too are applications. Most modern Linux distributions include what most would consider an app store. This is a centralized location where software can be searched and installed. Ubuntu Linux (and many other distributions) rely on GNOME Software, Elementary OS has the AppCenter, Deepin has the Deepin Software Center, openSUSE has their AppStore, and some distributions rely on Synaptic.

Regardless of the name, each of these tools do the same thing: a central place to search for and install Linux software. Of course, these pieces of software depend upon the presence of a GUI. For GUI-less servers, you will have to depend upon the command-line interface for installation.

Let’s look at two different tools to illustrate how easy even the command line installation can be. Our examples are for Debian-based distributions and Fedora-based distributions. The Debian-based distros will use the apt-get tool for installing software and Fedora-based distros will require the use of the yum tool. Both work very similarly. We’ll illustrate using the apt-get command. Let’s say you want to install the wget tool (which is a handy tool used to download files from the command line). To install this using apt-get, the command would like like this:

sudo apt-get install wget

The sudo command is added because you need super user privileges in order to install software. Similarly, to install the same software on a Fedora-based distribution, you would first su to the super user (literally issue the command su and enter the root password), and issue this command:

yum install wget

That’s all there is to installing software on a Linux machine. It’s not nearly as challenging as you might think. Still in doubt?

You can install a complete LAMP (Linux Apache MySQL PHP) server on either a server or desktop distribution. It really is that easy.

More resources

If you’re looking for one of the most reliable, secure, and dependable platforms for both the desktop and the server, look no further than one of the many Linux distributions. With Linux you can assure your desktops will be free of trouble, your servers up, and your support requests minimal.

Introduction to Bash - Learn the 5 Basic Principles of Bash; Comments, Variables, Functions, Loops, Conditional Statements

If you’ve ever used a Linux operating system used on most Virtual Private Servers, you may have heard of bash. It’s a Unix shell that reads and executes various commands.

What Is Bash?

Bash, short for Bourne-Again Shell, is a Unix shell and a command language interpreter. It reads shell commands and interacts with the operating system to execute them.

Why Use Bash Scripts?

Bash scripts can help with your workflow as they compile many lengthy commands into a single executable script file. For example, if you have multiple commands that you have to run at a specific time interval, you can compile a bash script instead of typing out the commands manually one by one. You then execute the script directly, when it’s necessary.

Pro Tip Linux has a bash shell command manual. Type man command to find descriptions of all the technical terms and input parameters.

Get Familiar With Bash Commands

Bash is available on almost all types of Unix-based operating systems and doesn’t require a separate installation. You will need a Linux command prompt, also known as the Linux terminal. On Windows you would use something like PuTTy. It’s a program that contains the shell and lets you execute bash scripts. 

1. Comments

Comments feature a description on certain lines of your script. The terminal doesn’t parse comments during execution, so they won’t affect the output.

There are two ways to add comments to a script. The first method is by typing # at the beginning of a single-line comment. # Command below prints a Hello World text echo “Hello, world!”

2. Variables

Variables are symbols that represent a character, strings of characters, or numbers. You only need to type the variable name in a command line to use the defined strings or numbers.

To assign a variable, type the variable name and the string value like here: testvar=“This is a test variable”

In this case, testvar is the variable name and This is a test variable is the string value. When assigning a variable, we recommend using a variable name that’s easy to remember and represents its value.

To read the variable value in the command line, use the $ symbol before the variable name. Take a look at the example below:

testvar=“This is a test variable”
echo $testvar

In order to let the user enter the variable contents use:

read testvar
echo $testvar

3. Functions

A function compiles a set of commands into a group. If you need to execute the command again, simply write the function instead of the whole set of commands.

There are several ways of writing functions. The first way is by starting with the function name and following it with parentheses and brackets:

function_name () {
    first command
    second command
}

Or, if you want to write it in a single line: function_name () { first command; second command; }

4. Loops

Loop bash commands are useful if you want to execute commands multiple times. There are three types of them you can run in bash – for, while, and until. The for loop runs the command for a list of items:

for item in [list]
do
    commands
done

The following example uses a for loop to print all the days of the week:

for days in Monday Tuesday Wednesday Thursday Friday Saturday Sunday
do
    echo “Day: $days”
done

On line 2, “days” automatically becomes a variable, with the values being the day names that follow. Then, in the echo command, we use the $ symbol to call the variable values.

The output of that script will be as follows:

Day: Monday
Day: Tuesday
Day: Wednesday
Day: Thursday
Day: Friday
Day: Saturday
Day: Sunday

Notice that even with just one command line in the loop script, it prints out seven echo outputs.

The next type of loop is while. The script will evaluate a condition. If the condition is true, it will keep executing the commands until the output no longer meets the defined condition.

while [condition]
    do
commands
done

5. Conditional Statements

Many programming languages, including bash, use conditional statements like if, then, and else for decision-making. They execute commands and print out outputs depending on the conditions. The if statement is followed by a conditional expression. After that, it’s followed by then and the command to define the output of the condition. The script will execute the command if the condition expressed in the if statement is true.

However, if you want to execute a different command if the condition is false, add an else statement to the script and follow it with the command.

Let’s take a look at simple if, then, and else statements. Before the statement, we will include a variable so the user can input a value:

echo “Enter a number”
read num
if [[$num -gt 10]]
then
echo “The number is greater than 10”
else
echo “The number is not greater than 10”

OK, so that's it. The 5 building blocks of Bash in plain English. Simple, right?!