Professional Documents
Culture Documents
The Linux Devops Handbook Customize and Scale Your Linux Distributions To Accelerate Your Devops Workflow 1O Edition Wojslaw Full Chapter
The Linux Devops Handbook Customize and Scale Your Linux Distributions To Accelerate Your Devops Workflow 1O Edition Wojslaw Full Chapter
Damian Wojsław
Grzegorz Adamowicz
BIRMINGHAM—MUMBAI
The Linux DevOps Handbook
Copyright © 2023 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted
in any form or by any means, without the prior written permission of the publisher, except in the case
of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information
presented. However, the information contained in this book is sold without warranty, either express
or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable
for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and
products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot
guarantee the accuracy of this information.
ISBN 978-1-80324-566-9
www.packtpub.com
To my wife, my kids, and all the people who have helped me on my journey to become the best version
of me.
– Damian Wojsław
To my mum and dad, who supported me with all my interests, including computers. To my wife and
daughter, who are my biggest supporters and motivation in all endeavors.
– Grzegorz Adamowicz
Contributors
I would like to thank my wife and kids for their patience and support.
Grzegorz Adamowicz has been working in the IT industry since 2006 in a number of positions,
including systems administrator, backend developer (PHP and Python), systems architect, and site
reliability engineer. Professionally, he is focused on building tools and automations for the projects
he is involved in. He also engages with the professional community by organizing events such as
conferences and workshops. Grzegorz has worked in many industries, including oil and gas, hotel,
Fintech, DeFI, automotive, and space.
I want to thank my wife, Agnieszka, for her never-ending support, and everyone I’ve had the pleasure
of working with throughout my career – most of what I know professionally, I’ve learned from you.
About the reviewer
Marcin Juszkiewicz is a seasoned software engineer on a journey since 2004. Over the years, he has
worked on building and optimizing code in various projects. His career has spanned from embedded
Linux systems to ARM servers. In his spare time, Marcin dons the hat of a system admin, a role he
embraces with the same enthusiasm as his daily job. Through his extensive journey as a software
engineer and part-time system administrator, Marcin has garnered a wealth of knowledge and a
reputation for his meticulous approach to coding and system optimization.
Table of Contents
Prefacexv
2
Command-Line Basics 15
Technical requirements 15 Interacting with the system 31
The Linux command line – shell 16 Process this 32
Getting to know your shell 16 Looking for something? 37
I call to thee 18 Let’s be manipulative 40
The filesystem structure 18 Summary42
Running a program 21
Exercises42
The command to teach you all commands 22
Know your environment 25
Resources43
Where in the PATH am I? 26
3
Intermediate Linux 45
Technical requirements 45 Adding users 59
Globs46 Modifying users 59
Removing users 60
Automating repetitive tasks 48
Managing groups 60
Cron jobs 48
Systemd timer 49 Secure Shell (SSH) protocol 61
Configuring OpenSSH 62
Software installation 49
Creating and managing SSH keys 64
Debian and Ubuntu 50
CentOS, RHEL, and Fedora 53 Summary66
Alpine Linux 54 Exercises66
Managing users 58
4
Automating with Shell Scripts 67
Technical requirements 67 Loops72
Backing up a database 67 Conditional execution – if statement 73
6
Networking in Linux 107
Networking in Linux 107 nftables132
ISO/OSI as a networking standard 108 ufw133
firewalld134
Physical layer 108
Data link layer – MAC, VLAN 109 Advanced topics 134
Network layer – IPv4 and IPv6 110 NAT134
Transport layer – TCP and UDP 118 Port forwarding 135
Session layer 122 Interface bonding 136
Presentation layer – SSL and TLS 123 TCP multipath 136
Application layer – HTTP and FTP 126 BGP137
Firewalls129 Summary138
iptables130
7
Git, Your Doorway to DevOps 139
Technical requirements 139 Branching150
Basic Git commands 140 Squashing commits using an interactive rebase 151
Solving Git conflicts 152
Configuring the local Git environment 141
Setting up a local Git repository 141 GitFlow and GitHub Flow 153
Local versus remote Git repositories 144 Global git configuration – .gitconfig 154
Ignoring some files using a .gitignore
Interacting with remote repositories 145
configuration file 156
What’s the git diff command? 147
Viewing the commit history 148 Summary158
8
Docker Basics 159
Technical requirements 159 Virtualization160
Virtualization versus containerization159 Containerization161
x Table of Contents
9
A Deep Dive into Docker 189
Docker advanced use cases 189 Multi-stage build 212
Running public images 190 ADD command use cases 213
Running a debugging container 191 Secrets management 214
Cleaning up unused containers 192 Docker orchestration 216
Docker volumes and bind mounts 194
Docker Swarm 216
Docker networking advanced use cases 195
Kubernetes and OpenShift 220
Security features of Docker 197
Docker for CI/CD pipeline integration 205 Summary221
Docker Compose 207
Advanced Dockerfile techniques 212
Table of Contents xi
Open source solutions for self-hosting 239 Log and metrics retention 248
Prometheus240 Full retention 249
Grafana241 Time-based retention 249
SigNoz242 Event-based retention 249
New Relic Pixie 242 Selective retention 249
Graylog243 Tiered retention 249
Sentry244 Summary250
11
Using Ansible for Configuration as Code 251
Technical requirements 251 Ansible Vault 277
CM systems and CaC 251 SOPS278
Other solutions 280
SaltStack253
Chef254 Ansible Tower and alternatives 280
Puppet256 Advanced topics 281
CFEngine257
Debugging281
Ansible258 Linting Ansible playbooks 283
Basics of using Ansible 261 Speeding up SSH connections 285
Tasks263 Dynamic inventory 286
Roles263 Summary287
Plays and playbooks 264
Further reading 287
Ansible Galaxy 275
Handling secrets 277
xii Table of Contents
12
Leveraging Infrastructure as Code 289
Technical requirements 290 Terraform state 302
What is IaC? 290 Terraform CLI 305
13
CI/CD with Terraform, GitHub, and Atlantis 335
Technical requirements 335 Deployment354
What is CI/CD? 336 CI/CD with Atlantis 356
An example of CI/CD pipelines 337 Deploying Atlantis to AWS 356
Continuously integrating and Running Terraform using Atlantis 362
deploying your infrastructure 342 Summary365
Integration testing 342 Exercises365
14
Avoiding Pitfalls in DevOps 367
Technical requirements 367 Tasks with version control integration 369
Too much or not enough automation 368 Tasks with repeatable patterns 369
Tasks with well-defined APIs or interfaces 369
Repetitive tasks 368
Tasks with clear and well-defined requirements370
Time-consuming tasks 369
Manual and error-prone tasks 369 Not understanding the technology 370
Table of Contents xiii
Index389
Chapter 3, Intermediate Linux, describes more advanced features of GNU/Linux that you will find useful.
Chapter 4, Automating with Shell Scripts explains how to start writing your own scripts using the
Bash shell.
Chapter 5, Managing Services in Linux, discusses different ways of managing services in Linux and
shows you how to define your own services using systemd.
Chapter 6, Networking in Linux, describes how networking works, how to control different aspects of
network configurations, and how to use command-line tools.
Chapter 7, Git, Your Doorway to DevOps, discusses what Git is and how to use Git’s version control
system, including less commonly known Git features.
Chapter 8, Docker Basics, explores the containerization of your services or tools, and running and
managing containers.
Chapter 9, A Deep Dive into Docker, discusses the more advanced features of Docker, including Docker
Compose and Docker Swarm.
Chapter 10, Monitoring, Tracing, and Distributed Logging, discusses how to monitor your services,
what tools you can use in the cloud, and how to do a basic setup.
Chapter 11, Using Ansible for Configuration as Code, looks at Configuration as Code with the use of
Ansible; it’ll guide you through the basic setup and more advanced features of Ansible.
Chapter 12, Leveraging Infrastructure as Code, discusses what Infrastructure as Code (IaC) is, what
the popular tools are, and how to manage your infrastructure using Terraform.
Chapter 13, CI/CD with Terraform, GitHub, and Atlantis, takes IaC one step further with the use of
the Continuous Integration (CI) and Continuous Deployment (CD) of infrastructure with the use
of Terraform and Atlantis.
Chapter 14, Avoiding Pitfalls in DevOps, discusses challenges you may encounter with your work
in DevOps.
This book is aimed at beginners in the DevOps world, and it assumes that you are eager to learn
about the tools and concepts that are commonly used in this field. By the end of this book, you will
have gained a solid understanding of how to manage infrastructure using IaC tools such as Terraform
and Atlantis, as well as how to automate repetitive tasks using Ansible and Bash scripting. You will
also learn how to set up logging and monitoring solutions to help you maintain and troubleshoot
your infrastructure.
Conventions used
There are a number of text conventions used throughout this book.
Code in text: Indicates code words in text, database table names, folder names, filenames, file
extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “While
logged in as root, your prompt will end with the # sign. When logged in as a normal user, it will
present you with $.”
A block of code is set as follows:
When we wish to draw your attention to a particular part of a code block, the relevant lines or items
are set in bold:
Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words
in menus or dialog boxes appear in bold. Here is an example: “Ansible Galaxy is a community-driven
platform that hosts an extensive collection of Ansible roles and playbooks.”
Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, email us at customercare@
packtpub.com and mention the book title in the subject of your message.
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen.
If you have found a mistake in this book, we would be grateful if you would report this to us. Please
visit www.packtpub.com/support/errata and fill in the form.
Piracy: If you come across any illegal copies of our works in any form on the internet, we would
be grateful if you would provide us with the location address or website name. Please contact us at
[email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you
are interested in either writing or contributing to a book, please visit authors.packtpub.com.
xix
https://packt.link/free-ebook/9781803245669
In this opening part of the book, we will focus on Linux distributions and the basic skills you will
need to efficiently use Linux operating systems. You will also learn about the basics of shell scripting
to automate day-to-day tasks.
This part has the following chapters:
Technical requirements
This chapter doesn’t have any technical requirements. We won’t run any commands or install any
software yet. This will come in the later chapters.
The code we’re presenting in the book is available in the public GitHub repository for your consideration
at this address: https://github.com/PacktPublishing/The-Linux-DevOps-Handbook.
Long before Linux was conceived by its creator, Linus Torvalds, there was Unix. Unix source code
was – for legal reasons – licensed to anyone who bought it, thus making it very popular among many
institutions. This included universities. The code, however, was not entirely free. This didn’t sit well
with many people, who believed that software should be free – as in speech or beer – including source
code. In the 1980s, a completely free and open implementation of Unix was born under the aegis of the
GNU Project. Its goal was to develop an operating system that gave complete control over a computer
to the user. The project was successful in that it was able to produce all the software required to run
an operating system, except one thing – the kernel.
A kernel of the operating system is, in short, the core that operates hardware and manages the hardware
and programs for the user.
In 1991, Finnish student Linus Torvalds famously announced his hobby kernel – Linux. He called it
at that time “just a hobby – won’t be big and professional like GNU.” It wasn’t supposed to get big and
popular. The rest is history. The Linux kernel became popular among developers and administrators
and became the missing piece of the GNU Project. Linux is the kernel, the GNU tools are the so-called
userland, and together they make the GNU/Linux operating system.
The preceding short story is important to us for two reasons:
• While the GNU userland and the Linux kernel is the most popular combination, you’ll see it
is not the only one.
• Linux delivers a kernel and the GNU Project delivers userland tools, but they have to be
somehow prepared for installation. Many people and teams had separate ideas on how to do
it best. I will expand on this thought next.
The way that a team or a company delivers a GNU/Linux operating system to end users is called a
distribution. It facilitates operating system installation, the means to manage software later on, and
general notions on how an operating system and the running processes have to be managed.
The RPM family of distributions starts with Red Hat Enterprise Linux (RHEL), created and maintained
by the Red Hat company. Closely related is Fedora (a free community distribution sponsored by Red
Hat). It also includes CentOS Linux (a free version of RHEL) and Rocky Linux (another free version
of RHEL).
The DEB distributions include Debian (where the DEB packages originate from) – a technocracy
community project. From the Debian distribution arose a number of distributions based on it, using
most of its core components. Most notable is Ubuntu, a server and desktop distribution sponsored
by Canonical.
There are also distributions that use packages with minimum automation, most notably Slackware,
one of the oldest existing Linux distributions.
There are distributions that give the user a set of scripts that compile actual software on the hardware
it will be used on – most notably, Gentoo.
Finally, there is a distribution that is actually a book with a set of instructions that a user can follow
to build the whole OS by hand – the Linux From Scratch project.
Another way of grouping distributions is by their acceptance of closed software – software that limits
the distribution of source code, binaries, or both. This can mean hardware drivers, such as the ones
for NVIDIA graphic cards, and user software, such as movie codecs that allow you to play streamed
media and DVD and Blu-Ray discs. Some distributions make it easy to install and use them, while
some make it more difficult, arguing that we should strive for all software to be open source and free
(as in speech and as in beer).
Yet another way to differentiate them is the security framework a given distribution uses. Two of the most
notable ones are AppArmor, used mainly by Ubuntu, and SELinux (from the USA’s National Security
Agency), used by, among others, Red Hat Enterprise Linux (and its derivatives) and Fedora Linux.
It’s also worth noting that while most Linux distributions use the GNU Project as the userland, the
popular one in the cloud Alpine Linux distribution uses its own set of software, written especially
with minimum size in mind.
Looking at how the distribution is developed, it can be community-driven (without any commercial
entity being an owner of any part of the process and software – Debian being one prime example),
commercial (wholly owned by a company – RHEL being one example and SuSE another), and all the
mixes in between (Ubuntu and Fedora being examples of a commercially owned distribution with a
large body of independent contributors).
6 Choosing the Right Linux Distribution
Finally, a way we can group distributions is by how well they facilitate the cloud workload. Here, we
can look at different aspects:
• The server side: How well a given distribution works as an underlying OS for our infrastructure,
virtual machines, and containers
• The service side: How well a given distribution is suited to run our software as a container or
a virtual machine
To make things even more confusing and amusing for new adopters, each distribution can have many
variants (called flavors, variants, or spins, depending on the distribution lingo) that offer different
sets of software or default configurations.
And to finally confuse you, dear reader, for use on a desktop or laptop, Linux offers the best it can
give you – a choice. The number of graphical interfaces for the Linux OS can spin the head of even
the most experienced user – KDE Plasma, GNOME, Cinnamon Desktop, MATE, Unity Desktop
(not related to the Unity 3D game engine), and Xfce. The list is non-exhaustive, subjective, and very
limited. They all differ in the ease of use, configurability, the amount of memory and CPU they use,
and many other aspects.
The number of distributions is staggering – the go-to site that tracks Linux distributions (https://
distrowatch.com/dwres.php?resource=popularity) lists 265 various Linux distributions
on its distribution popularity page at the time of writing. The sheer number of them makes it necessary
to limit the book to three of our choosing. For the most part, it doesn’t make a difference which one
you choose for yourself, except maybe in licensing and subscription if you choose a commercial one.
Each time the choice of distribution makes a difference, especially a technical one, we will point it out.
Choosing a distribution is more than just a pragmatic choice. The Linux community is deeply driven
by ideals. For some people, they are the most important ideals on which they build their lives. Harsh
words have been traded countless times over which text editor is better, based on their user interface, the
license they are published with, or the quality of the source code. The same level of emotion is displayed
toward the choice of software to run the WWW server or how to accept new contributions. This will
inevitably lead to the way the Linux distribution is installed, what tools there are for configuration
and maintenance, and how big the selection of software installed on it out of the box is.
Having said that, we have to mention that even though they have strong beliefs, the open-source
community, the Linux community included, is a friendly bunch. In most cases, you’ll be able to find
help or advice on online forums, and the chances are quite high that you will be able to meet them
in person.
What exactly is a Linux distribution? 7
• Is the software you wish to run supported on the distribution? Some commercial software
limits the number of distributions it publishes packages for. It may be possible to run them on
unsupported versions of Linux, but this may be tricky and prone to disruptions.
• Which versions of the software you intend to run are available? Sometimes, the distribution of
choice doesn’t update your desired packages often enough. In the world of the cloud, software
a few months old may already be outdated and lack important features or security updates.
• What is the licensing for the distribution? Is it free to use or does it require a subscription plan?
• What are your support options? For community-driven free distributions, your options are
limited to friendly Linux gurus online and in the vicinity. For commercial offerings, you can
pay for various support offerings. Depending on your needs and budget, you can find a mix of
support options that will suit your requirements and financial reserves.
• What is your level of comfort with editing configuration files and running long and complex
commands? Some distributions offer tools (both command-line and graphical) that make the
configuration tasks easier and less error-prone. However, those tools are mostly distribution-
specific, and you won’t find them anywhere else.
• How well are cloud-related tools supported on a given distribution? This can be the ease of
installation, the recency of the software itself, or the number of steps to configure for use.
• How well is this distribution supported by the cloud of your choosing? This will mean how
many cloud operators offer virtual machines with this distribution. How easy is it to obtain a
container image with this distribution to run your software in it? How easy do we suspect it to
be to build for this distribution and deploy on it?
• How well is it documented on the internet? This will not only include the documentation
written by distribution maintainers but also various blog posts and documentation (mainly
tutorials and so-called how-to documents) written by its users.
So far, you’ve learned what a Linux distribution is, how distributions differentiate from one another,
and what criteria you can use to actually choose one as the core of the system you will manage.
In the next section, we will look deeper into each distribution to get to know the most popular ones
better, giving you a first glimpse of how each one works and what to expect.
8 Choosing the Right Linux Distribution
Debian
Debian (https://www.debian.org/) is one of the oldest active Linux distributions. Its
development is led by the community-supported Debian Project. It is known for two things – the
sheer number of packages that the distribution provides and the slow release of stable versions. The
latter has improved in recent years and stable releases are published every two years. Software is
delivered in archives called packages. Debian packages’ names have a .deb file extension and are
colloquially called debs. They are kept online in repositories and repositories are broken down into
pools. Repositories offer almost 60,000 packages with software in the latest stable release.
Debian always has three versions available (so-called branches) – stable, testing, and unstable. The
releases are named after characters from the Toy Story movie franchise. The latest stable release –
version 11 – is called Bullseye.
The unstable branch is the rolling branch for developers, people who like living on the edge, or
those who require the newest software more than they require stability. Software is accepted into the
unstable branch with minimal testing.
The testing branch is where, as the name implies, the testing happens. A lot of testing happens here,
thanks to the end users. Packages come here from the unstable branch. The software here is still newer
than in the stable branch but not as fresh as in the unstable branch. A few months before the new
stable release, the testing branch is frozen. It means that no new software will be accepted, and new
versions of the already accepted packages are allowed only if they fix bugs.
After a few months, testing becomes the stable branch. The software is updated only for security fixes.
This distribution is available for many hardware platforms – Intel, ARM, PowerPC, and so on. Along
with unofficial ports, there are a multitude of hardware platforms on which you can install it.
Introducing the distributions 9
Debian is viewed as the most stable distribution there is, and it is used as a platform for various compute
clusters, so it is generally installed on bare-metal servers somewhere in a rack in a data center and
intended for use consistently over many years.
According to W3Techs (https://w3techs.com/technologies/details/os-linux),
Debian makes up for 16% of all servers running on the internet. Its derivative, Ubuntu, runs 33%
of them. Together, they account for 49% of all servers. This makes administration skills related to
Debian highly marketable.
Ubuntu Linux
The Ubuntu Linux distribution (https://ubuntu.com/) is widely credited for making Linux
popular on personal computers, and rightly so. Sponsored by Canonical, its mission was to make Linux
easily usable for most people. It was one of the first, if not the first, Linux versions to distribute non-free
and non-open binary drivers and libraries that made desktop use simpler and more comfortable.
Famously, the first bug report opened for Ubuntu distribution by Mark Shuttleworth (Canonical and
Ubuntu founder) was, “Microsoft has majority market share.”
The distribution itself is based on Debian Linux, and in the beginning, being fully binary-compatible
was one of the major objectives. As the development has progressed, this has lost some of its importance.
This distribution is developed by the community and Canonical. The main source of income for the
company is premium services related to Ubuntu Linux – support, training, and consultations.
Due to the very close-knit relationship between Debian Linux and Ubuntu Linux, many developers
and maintainers for one distribution serve the same roles in the other one. This results in a lot of
software being packaged for both distributions in parallel.
Ubuntu has three major flavors – Desktop, Server, and Core (for the internet of things). Desktop and
Server may differ slightly in how services are configured out of the box, and Core differs a lot
The software is distributed in .deb packages, the same as with Debian, and the sources are actually
imported from the Debian unstable branch. However, this doesn’t mean you can install Debian
packages on Ubuntu or vice versa, as they are not necessarily binary-compatible. It should be possible
to rebuild and install your own version.
There are four package repositories per release – the free and non-free software supported officially
by Canonical is called main and restricted, respectively. Free and non-free software delivered and
maintained by the community is called universe and multiverse, respectively.
10 Choosing the Right Linux Distribution
Important note
A word of advice – a widely accepted practice of system upgrades between major versions is to
wait for the first sub-release. So, if the currently installed version of the distribution is 2.5 and
the new version 3.0 is released, it is wise to wait until 3.1 or even 3.2 is released and upgrade
then. This is applicable to all the distributions we list here.
The Long-Term Support (LTS) versions are supported for five years. A new LTS version is released
every two years. It is possible to negotiate extended support. This gives a very good timeline to plan
major upgrades. A new Ubuntu version is released every six months.
Ubuntu Linux is widely adopted in education and government projects. Famously, the city of Munich,
between 2004 and 2013, migrated over 14,000 municipal desktop computers to a variant of Ubuntu
with the KDE desktop environment. While the migration saw disturbances politically – other operating
system vendors lobbied strongly against this migration – it was considered a success technically.
Ubuntu is the Linux of choice for personal computers. Canonical works very closely with hardware
vendors, notably Lenovo and Dell, but lately also with HP, to ensure full compatibility between the
distribution and the computers. Dell sells its flagship laptops with Ubuntu preinstalled.
Several sources cite Ubuntu Linux as the most installed Linux distribution on servers and personal
computers. The actual number can only be estimated, as Ubuntu doesn’t require any subscription
or registration.
As a byproduct of Ubuntu Linux’s popularity, software vendors, more often than not, offer .deb
packages of their software, if they release a Linux version. This is especially true for desktop software.
The amount of unofficial versions, clones, or modified distributions based on Ubuntu is staggering.
Ubuntu has a very active community, both organized and unorganized. It’s quite easy to get a hold of a
group of users near your city. This also directly translates to the amount of tutorials and documentation
on the internet.
Ubuntu Linux, especially under a support plan, is installed as a foundation for many cloud computing
infrastructure deployments. Many telecoms, banking, and insurance companies have chosen Ubuntu
Server as their foundation.
RHEL is considered a very stable and solid distribution. It is one of the main choices for banks, insurance
companies, and financial markets. It lacks many popular desktop software packages, but on the server
side of things, especially as an OS to run other commercial applications, it is a first-class citizen.
The software is distributed in online repository packages that end with .rpm, hence the name RPMs.
The main tool to administer the packages is RPM, with more sophisticated tools – yum, and lately
its successor, dnf – also available.
In the true spirit of an open source-based company, Red Hat makes sources for its distribution available.
This has led to the creation of a famous free and open clone of RHEL – CentOS. Until fairly recently,
it had been quite a popular choice for people who wanted to use RHEL but didn’t want to, or couldn’t,
pay a subscription. In 2014, CentOS joined the Red Hat company, and in 2020, Red Hat announced
that the versioned releases of CentOS would no longer be available; there would only be the so-called
rolling release, which constantly updates packages and does not mirror the RHEL releases. This resulted
in a very heated reaction from CentOS users. The original CentOS founder, Gregory Kurtzer, started
another clone of RHEL called Rocky Linux. Its main objective is the same as the original CentOS – to
deliver a free, open, and community-driven distribution, fully binary-compatible with RHEL.
The RHEL distribution delivers stable versions every few years and supports them for 10 years, starting
from release 5. The full support, however, is offered only for a few years. For the rest of the time, Red
Hat provides only security fixes and critical updates for your systems, with no new package versions
being introduced. Still, this life cycle is what users with large installations or mission-critical systems
came to like.
As with Ubuntu, it is possible to negotiate extended support time.
The Red Hat company has a turbulent relationship with the open source community. While the
company mostly plays fair, there have been some decisions that the community didn’t like. Lately, it
was Red Hat’s decision to change the CentOS release model to a rolling release (https://lists.
centos.org/pipermail/centos-announce/2020-December/048208.html).
RHEL, like Ubuntu, is the chosen foundation for commercially supported deployments of
cloud infrastructure.
Fedora Linux
Fedora (https://fedoraproject.org/wiki/Fedora_Project_Wiki) is a distribution
associated with the Red Hat company. While more than 50% of its developers and maintainers are
community members not affiliated with Red Hat, the company holds full stewardship over the
development. It is a RHEL upstream, which means that this is the real development frontend for the
actual RHEL. It doesn’t mean that everything from Fedora is included in the release of RHEL. However,
following Fedora closely will yield insight into the current direction of the RHEL distribution.
Contrary to RHEL, for which Fedora is the foundation, the new releases happen every six months.
It uses the same package type as RHEL, RPM.
12 Choosing the Right Linux Distribution
Fedora is considered a fast-paced distribution. It quickly adopts the newest and bleeding-edge versions
of packages.
CentOS
CentOS (https://centos.org) used to be the go-to free version of RHEL. The name is an acronym
for Community Enterprise Operating System. Its main goal was to be fully binary-compatible with
RHEL and adhere to the same releases and numbering scheme. In 2014, CentOS joined Red Hat, but
it was promised that the distribution would keep its independence from the company while benefiting
from development and testing resources. Unfortunately, in 2020, Red Hat announced that CentOS
8 would be the last numbered release, and from then on, CentOS Stream would be the only variant.
CentOS Stream is a midstream version. This means it sits in the middle between bleeding-edge and
fast-paced Fedora and stable and production-ready RHEL. The difference between CentOS Stream and
CentOS is that Stream is a development variant, while CentOS was simply a rebuilt and repackaged
mirror of the actual final product, RHEL.
All the knowledge, skills, and experience gained when working with RHEL are 100% applicable
to CentOS. Since CentOS is the third most-deployed Linux distribution on servers, according to
W3Techs (https://w3techs.com/technologies/details/os-linux), the skills are
very marketable.
Rocky Linux
As a response to the situation with the CentOS distribution, its founder announced the creation of
Rocky Linux (https://rockylinux.org/). The goals are the same as the original CentOS. The
release scheme and numbering follow RHEL. Shortly after the announcement, the GitHub repository
of Rocky Linux became top trending (https://web.archive.org/web/20201212220049/
https://github.com/trending). Rocky Linux is 100% binary-compatible with CentOS. The
project has released a set of tools that easily migrate from CentOS to Rocky Linux without reinstalling
the system.
The distribution is quite young, having been founded in 2020, and its popularity is still to be determined.
It has made a lot of noise in the community, and it seems that a steady stream of CentOS users have
moved to Rocky Linux as their preferred choice.
A very important contribution to the open source world from the Rocky Linux project is the build
system. It ensures that even if Rocky Linux shuts down, the community will be able to easily start up
a new RHEL clone.
All the skills, knowledge, and articles for RHEL and CentOS are 100% applicable to Rocky Linux. All
the software that runs on RHEL and CentOS should run without any modifications on Rocky Linux too.
Introducing the distributions 13
Alpine
Alpine Linux (https://alpinelinux.org/) is an interesting one. The main programming library
and most basic command-line tools are not from the GNU Project. Also, the services management
system, currently systemd in most distributions, is uncommon. This makes some of the administration
skills from other major distributions non-applicable. The strength of Alpine lies in its size (which
is rather small), its security-first mindset, and one of the fastest boot times among existing Linux
distributions. Those characteristics, with the boot time being admittedly more important, make it the
most popular choice for containers. If you run containerized software or build your own container
images, it is very likely that it is on Alpine Linux.
Alpine has its roots in the LEAF (Linux Embedded Appliance Framework; see: https://bering-
uclibc.zetam.org/wiki/Main_Page) project – a Linux distribution that fits on a single
floppy disk. LEAF is currently a popular choice for embedded markets, routers, and firewalls. Alpine
is a bigger distribution, but that sacrifice had to be made, since developers wanted to include several
useful but rather large software packages.
The package manager is called apk. The build system is borrowed from another distribution called
Gentoo Linux. As Gentoo builds software as it installs it, the portage obviously contains a lot of logic
around building software that is used as a part of an OS.
Alpine can run from RAM entirely. There’s even a special mechanism that allows you to initially
only load a few required packages from the boot device, and it can be achieved using Alpine’s Local
Backup Utility (LBU).
As mentioned before, this is a preferred distribution for container images. You won’t see it running
on a large server installation often, if at all. When we cross over to the cloud world, chances are you’ll
see a lot of Alpine Linux.
Having said that, every single one of those distributions has a variant for the cloud as a container base
image – a way to run your software in the true cloud way.
In this chapter, you learned the basics of popular Linux distributions and how they are different
from one another. You should now have some understanding of what you can choose from and what
consequences you will need to face – good and bad. To give you an even better idea of how to interact
with some cherry-picked Linux distributions, we will look at how to interact with a system using your
keyboard in Chapter 2.
14 Choosing the Right Linux Distribution
Summary
The short list in this chapter is just a tiny portion of the Linux distributions available. The list is largely
based on the popularity and marketability of skills, as well as our own experience and knowledge
that we acquired over the years. They are not, by any means, the best or only choices that you have.
We tried to point out where the main strengths lie and what a user’s relationship is with
respective distributions.
It’s not likely we were able to answer all your questions. Each of the Linux distributions from our list
has its own book out there, and there is even more knowledge on blogs, wikis, and YouTube tutorials.
In the next chapter, we will dive into the magical world of the command line.
2
Command-Line Basics
In this chapter, we’re going to dive right into the Linux command line. We will explain what makes
it so powerful and, by extension, important to every system administrator and DevOps person. But
more importantly, we will start teaching you the most useful commands and a way to use them
efficiently. Along the way, we will be adding other core Linux concepts, as they will be required to
understand the chapter.
In this chapter, we’re going to cover the following main topics:
It is not possible to introduce all the commands and tools in a single chapter. What follows is our
choice of the most basic tools you will need to know. Managing the Linux system is a separate book
topic on its own. It so happens that Packt does have several publications on that.
Technical requirements
It is highly recommended to have a Linux system installed and ready for use. We recommend it to be
a virtual machine or a laptop that you can safely reinstall from scratch in case something goes horribly
wrong. It will let you follow the examples in the book and perform any kind of exercise that we give you.
We are not going to cover an installation. Every distribution may use its own installer, either graphical
or text (depending on the distribution and which variant you’ve picked). You’ll need to note down
or remember the name of your user (conveniently called username or login) and password. There
are ways to get into the system if you have physical access and you don’t know either the login or
password, or both, but they are way outside of the scope of this book.
Our main distribution in the book is Debian. However, you should be alright with any of the major
ones we covered in the previous chapter, Choosing the Right Linux Distribution, as long as it isn’t Alpine.
16 Command-Line Basics
Note
You’ll also see the term console. There is a difference between the Terminal and a console. A
console is a physical device that lets users interact with the computer. It is the physical input
(nowadays, mostly keyboard) and output (nowadays, a monitor, but in the beginning, the
output was printed out). Terminal is a console emulator, a program that lets users perform
the same tasks.
The shell is a very powerful environment. It may seem cumbersome at first to do everything by typing
commands, but soon you’ll learn that the more complex the task, the more you can do with the shell,
more easily than with graphical interfaces. Every Linux system administrator worth their salt will
know how to do tasks in the shell and how to manage the system through it, and I wouldn’t risk much
by betting that they will prefer the shell to any GUI.
• sh: The original Steve Bourne shell. It is the shell, the very first one ever written that we know
of. While it lacks many interactive features that users came to appreciate from other, more
modern shells, sh is known for its speed of script execution and small size.
• ksh: Developed as an evolution of the Bourne shell, it is a superset of its predecessor. Thus, all
scripts written for sh will run in ksh.
The Linux command line – shell 17
• csh: The C shell. The name comes from its syntax, which closely follows the C programming language.
• zsh: The Z shell. It should be well known to macOS users, as it is a default on this operating
system. It is a completely modern shell, providing a lot of features that you’d expect from it:
command history, arithmetic operations, command completion, and so on.
We won’t trouble you much with shell variants and history. If you are interested in how Bash came to be,
refer to this Wikipedia article: https://en.wikipedia.org/wiki/Bash_(Unix_shell).
In our book, we are working with Bash. As mentioned earlier, it is the default shell for most Linux
distributions, it offers sh compatibility mode, it has all the features you’d expect from a modern shell,
and the amount of books, articles, how-tos, and other material for extending your knowledge about
it is staggering.
The first task that we will be performing is logging in to your shell. Depending on your chosen
installation method, you may need to power up the local virtual machine, your physical machine, or a
cloud-based Virtual Private Server (VPS). If you have picked a server installation without a graphical
user interface, you should see something similar to the following screenshot:
You are presented with a login screen, where you can provide your username and password. Once
successfully logged in, you are presented with a command prompt, which confirms that you’ve just
successfully started your shell.
The way the prompt looks is configurable and may be different depending on your chosen distribution.
There is, however, one thing that will stay the same, and we advise you to never change it. In the world
of Linux, there are two types of users: normal users and the superuser. While the login for normal users
can be anything as long as it adheres to the Linux user naming conventions, the superuser is called
root. To log in to the root account, you’d type in root as the username/login and its password next.
The superuser account is named that way for a reason. In most Linux distributions, it’s the omnipotent
user. Once logged in as root, you can do anything you wish, even deleting all files, including the
operating system itself.
18 Command-Line Basics
To help you distinguish between being a normal user and the root one, the prompt will present
you with a clue. While logged in as root, your prompt will end with the # sign. When logged in as
a normal user, it will present you with $.
While we’re at it, the # (hash) sign is also a so-called comment sign. If you happen to paste a command
or type it from the internet, if it starts with # or $, it is your clue as to which type of user should run this
command. You should omit this sign, especially as # in front will prevent the command from running.
In addition to the sign that ends the prompt, many distributions will prepend the username to it,
making sure you know which user you are. Given a user admin on a Linux system called myhome,
the default prompt for Debian 11 will look like this:
$admin@myhome:~$
root@myhome:~#
For the record, there are more ways to check your identity, but we will leave that for Chapter 4.
I call to thee
So far, so good. You have logged in, you know who you are, you can type, and you can read. But how
do you actually run programs?
In the Linux lingo, running a program is executing it or calling it. Actually, calling is mostly used
when referring to system commands or shell built-in commands, and executing is used when talking
about something that is not a part of the distribution—so-called third-party programs or binary.
Before I can tell you how to execute a program or call a command, I’ll have to explain a little bit about
filesystem structure and a very important system variable called PATH.
The process of making a network share or local drive available to the system or user is called mounting
and the resource that is made available is mounted. Remember that word, as we’re going to be using
it from now on. Different from Microsoft Windows, where your drives appear as a separate letter
with each being its own root directory, in Linux, your hard drive will be mounted somewhere within
the directories hierarchy. As an example, the user home directory can be stored on a separate hard
drive, mounted under the /home/ directory. You would never be able to tell it when browsing the
filesystem structure. The only way to tell would be to inspect mounted drives and partitions using
the following commands:
$ mount
oder
$ df
We are going to elaborate on them in Chapter 3, so for now, just know they exist.
The name of the uppermost directory is /. We already covered that. The separator between folders
nested in another folder is also /. So /usr/bin means a bin directory that exists in the usr
directory, and the usr directory exists in the / directory. Simple.
There is a very nice command that lets us inspect the directory structure, conveniently called tree.
It may not be present in your system. If so, don’t be worried; it’s not so important that you run it
rather than go through our explanation. In Chapter 3, when we introduce the installation of packages,
you can revisit and play around. By default, the tree command will flood your screen with the full
hierarchy, making it difficult to read and follow. There is an option, however, that lets us limit the
depth we will be inspecting:
admin@myhome:~$ tree -L 1 /
/
|-- bin -> usr/bin
|-- boot
|-- dev
|-- etc
|-- home
|-- lib -> usr/lib
|-- lib32 -> usr/lib32
|-- lib64 -> usr/lib64
|-- libx32 -> usr/libx32
|-- lost+found
|-- media
|-- mnt
|-- opt
|-- proc
|-- root
20 Command-Line Basics
|-- run
|-- sbin -> usr/sbin
|-- srv
|-- sys
|-- tmp
|-- usr
`-- var
22 directories, 0 files
There are several important concepts to cover here; we won’t be explaining all the directories at this
time, however. The first time a given folder becomes important, in Chapter 3 onward, we will briefly
touch upon it.
First, the calling of tree. You saw my prompt, which tells me I am running as user admin, on a
system named myhome and that I am not a root user (the dollar sign at the end). If you want to run
the tree command, you will skip the prompt. Next is the actual call: tree with the -L option and
number 1; this instructs the program to only print one level of depth. In other words, it will not go
any deeper into the directories. Finally, the / symbol tells the program to start printing at the very
beginning of the filesystem—the root folder.
Next, you’ll notice that some rows have this mysterious arrow pointing from one name to another.
This denotes a shortcut. There are two types of shortcuts, hard and symbolic. Directories can only
have a symbolic link. In the preceding output, the /bin directory is a link to a /usr/bin directory.
For all intents and purposes, they can be treated as one. There are technical and historical reasons
for the existence of this link. In days past, tools living in /bin and /sbin were used to mount a /
usr partition and then allow access to /usr/bin and /usr/sbin. Nowadays, this task is handled
earlier in the boot process by other tools and the requirement is no longer necessary. The structure is
kept for backward compatibility with tools that may require the existence of both /bin and /sbin
directories. More details can be found at https://refspecs.linuxfoundation.org/
FHS_3.0/fhs/index.html or https://www.linuxfoundation.org/blog/blog/
classic-sysadmin-the-linux-filesystem-explained.
Since we already touched on the /bin and /sbin directory, let’s explain the difference. The /usr/
bin directory contains binaries—in other words, commands that can be of interest to every user
in the system. The /usr/sbin directory contains so-called system binaries. Those are commands
that should concern only the root user. It will also contain binaries for system processes (called
daemons)—programs that run in the background and do important work for the running system.
The /root directory is the home of the superuser. This is where all its configuration files lay.
The interesting one is the /home directory. This is where all the user home directories exist. When I
created my admin user for my home machine, it was placed in the /home/admin folder.
The Linux command line – shell 21
Of importance to us at this moment will also be the /etc/ directory. It contains all the configuration
files for the whole system: the source for online package repositories, default shell configuration, system
name, processes that start at the boot, system users and their passwords, and time-based commands.
On a freshly installed Debian 11 system, the /etc/ directory contains about 150 subdirectories and
files, and each subdirectory may contain more folders inside.
The /tmp folder contains temporary files. They only live when the system is booted and will be
deleted the moment it is shut down or restarted. The nature of those files is often very volatile; they
can hop into existence and disappear very fast or can be modified very often. It is not uncommon for
this directory to only exist in the computer’s RAM. It’s the fastest storage device your system has and
will automatically purge on restart or power off.
As mentioned earlier, additional drives are mounted under this structure. We could have a separate
hard drive for home directories. The whole /home folder may live on it or even on a networked
hard disk array. As mentioned, the /tmp directory is often mounted in the RAM. Sometimes, the /
var directory (which contains things that can change often in the system but are not supposed to
be purged, such as logs) is mounted on a separate drive. One of the reasons is that the contents of /
var, and especially of /var/log (where the system logs live), can grow very fast and take all the
available space, making accessing the system impossible or very tricky.
Finally, there are two important and special directories that exist everywhere you go:
• .: A single dot means the current directory. If you choose to go to a . folder, you’ll end up
where you are. It’s useful, however, as you’ll see in Chapter 3.
• ..: Two dots mean the directory above us—the parent directory. If you choose to go to a ..
folder, you’ll end up one level above where you started. Note that for the / directory, both .
and .. mean the same: /. You can’t go any higher than the root.
Running a program
Now that we know some things about the folder hierarchy, let’s briefly touch on executing programs.
There are three basic ways to execute a program in a shell:
• Call it by its name: This requires that it is placed in a directory that is in a PATH variable
(explained in Chapter 3).
• Call it by an absolute path: The absolute path starts with /. When using the absolute path, you
have to list all the directories that lead to the program, including the leading /. An example
execution may look like this:
/opt/somecompany/importantdir/bin/program
22 Command-Line Basics
• Call it by a relative path: This will not start with a dot or two dots. This way of executing a
program requires that you know where in the filesystem you are and where in the filesystem
this program is. It’s useful if you are in the same directory as the binary or very close to it:
To call a program in the same directory as your user, you precede it with a dot and a slash:
./. This shortcut means the current directory. A sample call could look like ./myprogram
or ./bin/myprogram. The latter would mean: let’s start a program called myprogram
that is in a bin directory that is in the current directory.
To call a program in a directory somewhere else in the system using a relative path, we will
have to use the two dots, meaning the parent folder. Let’s say you are logged in to your home
directory, /home/admin, and want to execute a program in /opt/some/program/
bin; you’d call ../../opt/some/program/bin/myprogram. The two dots and
slash mean moving up.
If this looks arcane, it is because it is a little bit. Fortunately, everything will start coming together as
the book progresses.
$ man
The man command is a shorthand for manual and it is exactly what it says: it is a manual for whatever
command you want to learn about. To learn more about the man command, simply call the following:
$ man man
MAN(1)
Manual pager utils
MAN(1)
NAME
man - an interface to the system reference manuals
SYNOPSIS
man [man options] [[section] page ...] ...
man -k [apropos options] regexp ...
The Linux command line – shell 23
DESCRIPTION
man is the system's manual pager. Each page argument given
to man is normally the name of a program, utility or function. The
manual page associated with each of these arguments is then found and
displayed. A section, if
provided, will direct man to look only in that section of the
manual. The default action is to search in all of the available
sections following a pre-defined order (see DEFAULTS), and to show
only the first page found,
even if page exists in several sections.
I have cut it for brevity. A well-written man page will have several sections:
• name: This is where the name of the command is stated. If the command exists under various
names, they will all be listed here.
• synopsis: This will list possible ways of calling the command.
• description: This is the purpose of the command.
• examples: This will show several examples of the command call, to make the syntax clearer
and give some ideas.
• options: This will show all the available options and their meaning.
• getting help: This is how to get a shortened version of the command synopsis.
• files: If the command has configuration files or consumes files known to exist in the
filesystem, they will be listed here (for man, I have listed /etc/manpath.config and /
usr/share/man).
• bugs: This is where to look for bugs and report new ones.
• history: This will show all the current and previous authors of the program.
• see also: These are programs that are in some way tied to the command (for man:
apropos(1), groff(1), less(1), manpath(1), nroff(1), troff(1), whatis(1),
zsoelim(1), manpath(5), man(7), catman(8), and mandb(8)).
Many commands will contain a lot of additional sections, specific to this program.
24 Command-Line Basics
The man pages will contain a lot of knowledge, and sometimes the names will duplicate. This is where
the mysterious numbers in brackets come into play. The man pages are broken down into sections.
To quote the man page about man:
Let’s take, for example, printf. There are several things that are called printf. One of them is a
library function of the C programming language. Its man page will live in section 3.
To read about that library function, you have to tell man to look into section 3:
NAME
printf, fprintf, dprintf, sprintf, snprintf, vprintf, vfprintf,
vdprintf, vsprintf, vsnprintf - formatted output conversion
If you don't, what you'll get is a shell function for printing—totally
useless in C programming:
admin@myhome:~$ man printf
PRINTF(1)
User Commands
PRINTF(1)
NAME
printf - format and print data
SYNOPSIS
printf FORMAT [ARGUMENT]...
printf OPTION
Another random document with
no related content on Scribd:
Turk and Armenian are in the wrong, and that, as very often
happens, it is the innocent who have suffered for the
wrong-doings of the guilty. When it is asserted on behalf of
the Turks that they are engaged in suppressing a revolutionary
movement in Armenia, the statement is fully justified by the
facts of the case. There does exist in Armenia an extremely
vigorous revolutionary movement, and it is equally beyond
question that the methods of some of the leaders of this
movement are no less shocking than the barbarity of the Turk
in suppressing it. At every step," he added, "I became more
and more convinced that the inhuman ferocity displayed in this
terrible struggle for the mastery has not been in the least
exaggerated in the reports of the massacres already published
in England. At Bitlis I heard the story of a Turkish soldier
who boasted, as one who had achieved a glorious feat, that he
had taken part in the disembowelling of thirty pregnant
women. 'Two lives in one,' was the rallying cry of the armed
men who perpetrated this butchery. Another soldier, who had
taken part in a massacre in a church, described, gloating upon
every ghastly detail, how he had slipped and slid along the
blood-washed floor while the inhuman work proceeded.
Unfortunately, something very like a counterpart of these
atrocities is presented by the methods of some of the leaders
of the Armenian revolutionary movement. I believe there is no
doubt of the fact that certain of these Armenian conspirators
arranged to murder the Reverend Dr. Edward Riggs and two other
American missionaries at Marsovan, and fasten the blame upon
the Turks, in order that, as they imagined, the United States
might inflict summary punishment upon the Turkish Government,
thereby rendering Armenian independence possible. The
missionaries only escaped through a timely warning which they
received from an Armenian friend. Dr. Riggs has devoted his
life to the education of the Armenian youth in the missionary
schools, but the conspirators, in their blind fanaticism, gave
this fact little heed."
{539}
TURKEY: A. D. 1896.
Conflict in Crete between Christians and Mussulmans,
and its preceding causes.
In 1868, the Cretans, for the second time, were thrust under
the Turkish yoke. "By way of solace the Powers exerted
themselves feebly in inducing the Porte to concede the
so-called 'Organic Statute'
Ypsiloritis,
The Situation in Crete
(Contemporary Review, September, 1896).
E. J. Dillon,
Crete and the Cretans
(Fortnightly Review, May, 1897).
"The way is all made clear for sending supplies. The suitable
agents all along the route are now known, and have been
arranged with for service, so that heavy supplies can be sent
at any and all times as they are needed. I feel my breath come
lighter as I think of these poor scourged and fever-stricken
towns without even one doctor, when our sixteen strong,
skilled men, with twenty-five camels' burden of supplies,
shall carry some light of hope and help into their night of
hopeless woe. I am happy to be able to say for the comfort of
contributors, that I hold the written word of the Porte,
officially given through the minister of foreign affairs from
the grand vizier, that not the slightest interference with any
distribution within the province will be had. This official
document was addressed and delivered to Sir Philip Currie, the
British ambassador, and by him passed to me. The decision is
general and final, without question or reservation, and
settles all doubt."
{542}
"Revolutionists are the same all the world over, but the
Turkish Government is unique, and it is not the attack on the
Bank which interests us but the action of the Government which
followed it. As we have said, the authorities had full
information of what was to be attempted and did nothing to
prevent it, but they made every preparation for carrying out
their own plan. Bands of ruffians were gathered in Stamboul,
Galata, and Pera, made up of Kurds, Lazes, and the lower class
of Turks, armed with clubs, knives or firearms; and care was
taken that no one should kill or plunder in the quarter to
which he belonged, lest he should be recognised and complaint
made afterwards by the Embassies, with a demand for
punishment. A large number of carts were in readiness to carry
off the dead. The troops and police were in great force to
prevent any resistance, and to assist the mob if necessary. It
was a beautiful day, the streets were crowded, and few had any
idea of what had happened at the Bank, when suddenly, without
any warning, the work of slaughter and plunder began,
everywhere at once. European ladies on the way to the
Bosphorus steamers suddenly found themselves surrounded by
assassins, and saw men beaten to death at their feet. Foreign
merchants saw their own employés cut to pieces at their doors.
The streets in some places literally ran with blood. Every man
who was recognised as an Armenian was killed without mercy. In
general, the soldiers took no part in the slaughter and
behaved well, and this somewhat reassured those in the streets
who were not Armenians; but in a few moments the shops were
closed and a wild panic spread through the city. The one idea
of everyone was to get home; and as the foreigners and better
classes live out of the city in summer they had to go to the
Galata bridge to take the steamers, which ran as usual all
through the three days of massacre. This took them through the
streets where the slaughter was going on, and consequently we
have the testimony of hundreds of eye-witnesses as to what
took place. The work of death and plunder continued unchecked
for two days. On Friday there were isolated outbreaks, and
occasional assassinations occurred up to Tuesday.
{543}