Machine Learning Infrastructure and

Best Practices for Software Engineers:

Take your machine learning software
from a prototype to a fully fledged
software system 1st Edition
Anonymous
Visit to download the full and correct content document:
https://ebookmeta.com/product/machine-learning-infrastructure-and-best-practices-for
-software-engineers-take-your-machine-learning-software-from-a-prototype-to-a-fully-f
ledged-software-system-1st-edition-anonymous/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Applied Machine Learning and AI for Engineers Jeff

Prosise

https://ebookmeta.com/product/applied-machine-learning-and-ai-
for-engineers-jeff-prosise/

Machine Learning Crash Course for Engineers 1st Edition

Eklas Hossain

https://ebookmeta.com/product/machine-learning-crash-course-for-
engineers-1st-edition-eklas-hossain/

Software-Defined Radio for Engineers Travis F Collins

https://ebookmeta.com/product/software-defined-radio-for-
engineers-travis-f-collins/

Engineer Your Software! 1st Edition Scott A. Whitmire

https://ebookmeta.com/product/engineer-your-software-1st-edition-
scott-a-whitmire/
Machine Learning Interviews: Kickstart Your Machine
Learning and Data Career 1st Edition Susan Shu Chang

https://ebookmeta.com/product/machine-learning-interviews-
kickstart-your-machine-learning-and-data-career-1st-edition-
susan-shu-chang/

Robotics for Software Engineers - MEAP Version 3

Andreas Bihlmaier

https://ebookmeta.com/product/robotics-for-software-engineers-
meap-version-3-andreas-bihlmaier/

Machine Learning Simplified A Gentle Introduction to

Supervised Learning Andrew Wolf

https://ebookmeta.com/product/machine-learning-simplified-a-
gentle-introduction-to-supervised-learning-andrew-wolf/

Practical Machine Learning for Computer Vision: End-to-

End Machine Learning for Images 1st Edition Valliappa
Lakshmanan

https://ebookmeta.com/product/practical-machine-learning-for-
computer-vision-end-to-end-machine-learning-for-images-1st-
edition-valliappa-lakshmanan/

Machine Learning System Design (MEAP V03) Valerii

Babushkin

https://ebookmeta.com/product/machine-learning-system-design-
meap-v03-valerii-babushkin/
Machine Learning Infrastructure and Best
Practices for Software Engineers
Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval
system, or transmitted in any form or by any means, without the prior written
permission of the publisher, except in the case of brief quotations embedded in
critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy
of the information presented. However, the information contained in this book is
sold without warranty, either express or implied. Neither the author nor Packt
Publishing or its dealers and distributors, will be held liable for any damages
caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Niranjan Naikwadi

Publishing Product Manager: Yasir Ali Khan

Book Project Manager: Hemangi Lotlikar

Senior Editor: Sushma Reddy

Technical Editor: Kavyashree K S

Copy Editor: Safis Editing

Proofreader: Safis Editing

Indexer: Hemangini Bari

Production Designer: Gokul Raj S.T

DevRel Marketing Coordinator: Vinishka Kalra

First published: January 2024

Production reference: 1170124

Published by

Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK

ISBN 978-1-83763-406-4

www.packtpub.com
Writing a book with a lot of practical examples requires a lot of extra time,
which is often taken from family and friends. I dedicate this book to my family
– Alexander, Cornelia, Viktoria, and Sylwia – who always supported and
encouraged me, and to my parents and parents-in-law, who shaped me to be
who I am.
 – Miroslaw Staron

Contributors

About the author

Miroslaw Staron is a professor of Applied IT at the University of Gothenburg in
Sweden with a focus on empirical software engineering, measurement, and
machine learning. He is currently editor-in-chief of Information and Software
Technology and co-editor of the regular Practitioner’s Digest column of IEEE
Software. He has authored books on automotive software architectures, software
measurement, and action research. He also leads several projects in AI for
software engineering and leads an AI and digitalization theme at Software Center.
He has written over 200 journal and conference articles.

I would like to thank my family for their support in writing this book. I would also
like to thank my colleagues from the Software Center program who provided me
with the ability to develop my ideas and knowledge in this area – in particular,
Wilhelm Meding, Jan Bosch, Ola Söder, Gert Frost, Martin Kitchen, Niels Jørgen
Strøm, and several other colleagues. One person who really ignited my interest in
this area is of course Mirosław “Mirek” Ochodek, to whom I am extremely grateful.
I would also like to thank the funders of my research, who supported my studies
throughout the years. I would like to thank my Ph.D. students, who challenged me
and encouraged me to always dig deeper into the topics. I’m also very grateful to
the reviewers of this book – Hongyi Zhang and Sushant K. Pandey, who provided
invaluable comments and feedback for the book. Finally, I would like to extend my
gratitude to my publishing team – Hemangi Lotlikar, Sushma Reddy, and Anant
Jaint – this book would not have materialized without you!

About the reviewers

Hongyi Zhang is a researcher at Chalmers University of Technology with over five
years of experience in the fields of machine learning and software engineering.
Specializing in machine learning, edge/cloud computing, and software engineering,
his research merges machine learning theory and software applications, driving
tangible improvements in industrial machine learning ecosystems.

Sushant Kumar Pandey is a dedicated post-doctoral researcher at the

Department of CSE, Chalmers at the University of Gothenburg, Sweden, who
seamlessly integrates academia with industry, collaborating with Volvo Cars in
Gothenburg. Armed with a Ph.D. in CSE from the esteemed Indian Institute of
Technology (BHU), India, Sushant specializes in the application of AI in software
engineering. His research advances technology’s transformative potential. As a
respected reviewer for prestigious venues such as IST, KBS, EASE, and ESWA,
Sushant actively contributes to shaping the discourse in his field. Beyond research,
he leverages his expertise to mentor students, fostering innovation and excellence
in the next generation of professionals.
Table of Contents

Preface
Part 1: Machine Learning Landscape in Software
Engineering

Machine Learning Compared to Traditional Software

Machine learning is not traditional software
Supervised, unsupervised, and reinforcement
learning – it is just the beginning
An example of traditional and machine learning
software
Probability and software – how well they go
together
Testing and evaluation – the same but different
Summary
References

Elements of a Machine Learning System

Elements of a production machine learning
system
Data and algorithms
Data collection
Feature extraction
Data validation
Configuration and monitoring
Configuration
Monitoring
Infrastructure and resource management
Data serving infrastructure
Computational infrastructure
How this all comes together – machine learning
pipelines
References

Data in Software Systems – Text, Images, Code, and

Their Annotations

Raw data and features – what are the

differences?
Images
Text
Visualization of output from more advanced text
processing
Structured text – source code of programs
Every data has its purpose – annotations and
tasks
Annotating text for intent recognition
Where different types of data can be used
together – an outlook on multi-modal data
models
References

Data Acquisition, Data Quality, and Noise

Sources of data and what we can do with them
Extracting data from software engineering tools
– Gerrit and Jira
Extracting data from product databases – GitHub
and Git
Data quality
Noise
Summary
References

Quantifying and Improving Data Properties

Feature engineering – the basics
Clean data
Noise in data management
Attribute noise
Splitting data
How ML models handle noise
References
Part 2: Data Acquisition and Management

Processing Data in Machine Learning Systems

Numerical data
Summarizing the data
Diving deeper into correlations
Summarizing individual measures
Reducing the number of measures – PCA
Other types of data – images
Text data
Toward feature engineering
References

Feature Engineering for Numerical and Image Data

Feature engineering
Feature engineering for numerical data
PCA
t-SNE
ICA
Locally linear embedding
Linear discriminant analysis
Autoencoders
Feature engineering for image data
Summary
References

Feature Engineering for Natural Language Data

Natural language data in software engineering
and the rise of GitHub Copilot
What a tokenizer is and what it does
Bag-of-words and simple tokenizers
WordPiece tokenizer
BPE
The SentencePiece tokenizer
Word embeddings
FastText
From feature extraction to models
References
Part 3: Design and Development of ML Systems

Types of Machine Learning Systems – Feature-Based

and Raw Data-Based (Deep Learning)
Why do we need different types of models?
Classical machine learning models
Convolutional neural networks and image
processing
BERT and GPT models
Using language models in software systems
Summary
References

10

Training and Evaluating Classical Machine Learning

Systems and Neural Networks
Training and testing processes
Training classical machine learning models
Understanding the training process
Random forest and opaque models
Training deep learning models
Misleading results – data leaking
Summary
References

11

Training and Evaluation of Advanced ML Algorithms –

GPT and Autoencoders

From classical ML to GenAI

The theory behind advanced models – AEs and
transformers
AEs
Transformers
Training and evaluation of a RoBERTa model
Training and evaluation of an AE
Developing safety cages to prevent models from
breaking the entire system
Summary
References

12

Designing Machine Learning Pipelines (MLOps) and

Their Testing
What ML pipelines are
ML pipelines
Elements of MLOps
ML pipelines – how to use ML in the system in
practice
Deploying models to HuggingFace
Downloading models from HuggingFace
Raw data-based pipelines
Pipelines for NLP-related tasks
Pipelines for images
Feature-based pipelines
Testing of ML pipelines
Monitoring ML systems at runtime
Summary
References

13

Designing and Implementing Large-Scale, Robust ML

Software

ML is not alone
The UI of an ML model
Data storage
Deploying an ML model for numerical data
Deploying a generative ML model for images
Deploying a code completion model as an
extension
Summary
References
Part 4: Ethical Aspects of Data Management and ML
System Development

14

Ethics in Data Acquisition and Management

Ethics in computer science and software
engineering
Data is all around us, but can we really use it?
Ethics behind data from open source systems
Ethics behind data collected from humans
Contracts and legal obligations
References

15

Ethics in Machine Learning Systems

Bias and ML – is it possible to have an objective

AI?
Measuring and monitoring for bias
Other metrics of bias
Developing mechanisms to prevent ML bias from
spreading throughout the system
Summary
References

16

Integrating ML Systems in Ecosystems

Ecosystems
Creating web services over ML models using
Flask
Creating a web service using Flask
Creating a web service that contains a pre-
trained ML model
Deploying ML models using Docker
Combining web services into ecosystems
Summary
References

17

Summary and Where to Go Next

To know where we’re going, we need to know
where we’ve been
Best practices
Current developments
My view on the future
Final remarks
References

Index

Other Books You May Enjoy

Preface
Machine learning has gained a lot of popularity in recent years. The introduction of
large language models such as GPT-3 and 4 only increased the speed of the
development of this field. These large language models have become so powerful
that it is almost impossible to train them on a local computer. However, this is not
necessary at all. These language models provide the ability to create new tools
without the need to train them because they can be steered by the context
window and the prompt.

In this book, my goal is to show how machine learning models can be trained,
evaluated, and tested – both in the context of a small prototype and in the context
of a fully-fledged software product. The primary objective of this book is to bridge
the gap between theoretical knowledge and practical implementation of machine
learning in software engineering. It aims to equip you with the skills necessary to
not only understand but also effectively implement and innovate with AI and
machine learning technologies in your professional pursuits.

The journey of integrating machine learning into software engineering is as

thrilling as it is challenging. As we delve into the intricacies of machine learning
infrastructure, this book serves as a comprehensive guide, navigating through the
complexities and best practices that are pivotal for software engineers. It is
designed to bridge the gap between the theoretical aspects of machine learning
and the practical challenges faced during implementation in real-world scenarios.

We begin by exploring the fundamental concepts of machine learning, providing a

solid foundation for those new to the field. As we progress, the focus shifts to the
infrastructure – the backbone of any successful machine learning project. From
data collection and processing to model training and deployment, each step is
crucial and requires careful consideration and planning.

A significant portion of the book is dedicated to best practices. These practices are
not just theoretical guidelines but are derived from real-life experiences and case
studies that my research team discovered during our work in this field. These best
practices offer invaluable insights into handling common pitfalls and ensuring the
scalability, reliability, and efficiency of machine learning systems.

Furthermore, we delve into the ethics of data and machine learning algorithms.
We explore the theories behind ethics in machine learning, look closer into the
licensing of data and models, and finally, explore the practical frameworks that can
quantify bias in data and models in machine learning.

This book is not just a technical guide; it is a journey through the evolving
landscape of machine learning in software engineering. Whether you are a novice
eager to learn, or a seasoned professional seeking to enhance your skills, this
book aims to be a valuable resource, providing clarity and direction in the exciting
and ever-changing world of machine learning.
Who this book is for
This book is meticulously crafted for software engineers, computer scientists, and
programmers who seek practical applications of artificial intelligence and machine
learning in their field. The content is tailored to impart foundational knowledge on
working with machine learning models, viewed through the lens of a programmer
and system architect.

The book presupposes familiarity with programming principles, but it does not
demand expertise in mathematics or statistics. This approach ensures accessibility
to a broader range of professionals and enthusiasts in the software development
domain. For those of you without prior experience in Python, this book
necessitates acquiring a basic understanding of the language. However, the
material is structured to facilitate a rapid and comprehensive grasp of Python
essentials. Conversely, for those proficient in Python but not yet seasoned in
professional programming, this book serves as a valuable resource for transitioning
into the realm of software engineering with a focus on AI and ML applications.
What this book covers
Chapter 1, Machine Learning Compared to Traditional Software, explores where
these two types of software systems are most appropriate. We learn about the
software development processes that programmers use to create both types of
software and we also learn about the classical four types of machine learning
software – rule-based, supervised, unsupervised, and reinforcement learning.
Finally, we also learn about the different roles of data in traditional and machine
learning software.

Chapter 2, Elements of a Machine Learning System, reviews each element of a

professional machine learning system. We start by understanding which elements
are important and why. Then, we explore how to create such elements and how to
work by putting them together into a single machine learning system – the so-
called machine learning pipeline.

Chapter 3, Data in Software Systems – Text, Images, Code, and Features,

introduces three data types – images, texts, and formatted text (program source
code). We explore how each of these types of data can be used in machine
learning, how they should be annotated, and for what purpose. Introducing these
three types of data provides us with the possibility to explore different ways of
annotating these sources of data.

Chapter 4, Data Acquisition, Data Quality, and Noise, dives deeper into topics
related to data quality. We go through a theoretical model for assessing data
quality and we provide methods and tools to operationalize it. We also look into
the concept of noise in machine learning and how to reduce it by using different
tokenization methods.

Chapter 5, Quantifying and Improving Data Properties, dives deeper into the
properties of data and how to improve them. In contrast to the previous chapter,
we work on feature vectors rather than raw data. The feature vectors are already
a transformation of the data; therefore, we can change such properties as noise or
even change how the data is perceived. We focus on the processing of text, which
is an important part of many machine learning algorithms nowadays. We start by
understanding how to transform data into feature vectors using simple algorithms,
such as bag of words, so that we can work on feature vectors.

Chapter 6, Processing Data in Machine Learning Systems, dives deeper into the
ways in which data and algorithms are entangled. We talk a lot about data in
generic terms, but in this chapter, we explain what kind of data is needed in
machine learning systems. We explain the fact that all kinds of data are used in
numerical form – either as a feature vector or as more complex feature matrices.
Then, we will explain the need to transform unstructured data (e.g., text) into
structured data. This chapter will lay the foundations for going deeper into each
type of data, which is the content of the next few chapters.

Chapter 7, Feature Engineering for Numerical and Image Data, focuses on the
feature engineering process for numerical and image data. We start by going
through the typical methods such as Principal Component Analysis (PCA),
which we used previously for visualization. We then move on to more advanced
methods such as the t-Student Distribution Stochastic Network
Embeddings (t-SNE) and Independent Component Analysis (ICA). What
we end up with is the use of autoencoders as a dimensionality reduction technique
for both numerical and image data.

Chapter 8, Feature Engineering for Natural Language Data, explores the first steps
that made the transformer (GPT) technologies so powerful – feature extraction
from natural language data. Natural language is a special kind of data source in
software engineering. With the introduction of GitHub Copilot and ChatGPT, it
became evident that machine learning and artificial intelligence tools for software
engineering tasks are no longer science fiction.

Chapter 9, Types of Machine Learning Systems – Feature-Based and Raw Data-

Based (Deep Learning), explores different types of machine learning systems. We
start from classical machine learning models such as random forest and we move
on to convolutional and GPT models, which are called deep learning models. Their
name comes from the fact that they use raw data as input and the first layers of
the models include feature extraction layers. They are also designed to
progressively learn more abstract features as the input data moves through these
models. This chapter demonstrates each of these types of models and progresses
from classical machine learning to the generative AI models.

Chapter 10, Training and Evaluation of Classical ML Systems and Neural Networks,
goes a bit deeper into the process of training and evaluation. We start with the
basic theory behind different algorithms and then we show how they are trained.
We start with the classical machine learning models, exemplified by the decision
trees. Then, we gradually move toward deep learning where we explore both the
dense neural networks and some more advanced types of networks.

Chapter 11, Training and Evaluation of Advanced ML Algorithms – GPT and

Autoencoders, explores how generative AI models work based on GPT and
Bidirectional Encoder Representation Transformers (BERT). These models are
designed to generate new data based on the patterns that they were trained on.
We also look at the concept of autoencoders, where we train an autoencoder to
generate new images based on the previously trained data.

Chapter 12, Designing Machine Learning Pipelines and their Testing, describes how
the main goal of MLOps is to bridge the gap between data science and operations
teams, fostering collaboration and ensuring that machine learning projects can be
effectively and reliably deployed at scale. MLOps helps to automate and optimize
the entire machine learning life cycle, from model development to deployment and
maintenance, thus improving the efficiency and effectiveness of ML systems in
production. In this chapter, we learn how machine learning systems are designed
and operated in practice. The chapter shows how pipelines are turned into a
software system, with a focus on testing ML pipelines and their deployment at
Hugging Face.

Chapter 13, Designing and Implementation of Large-Scale, Robust ML Software,

explains how to integrate the machine learning model with a graphical user
interface programmed in Gradio and storage in a database. We use two examples
of machine learning pipelines – an example of the model for predicting defects
from our previous chapters and a generative AI model to create pictures from a
natural language prompt.
Chapter 14, Ethics in Data Acquisition and Management, starts by exploring a few
examples of unethical systems that show bias, such as credit ranking systems that
penalize certain minorities. We also explain the problems with using open source
data and revealing the identities of subjects. The core of the chapter, however, is
the explanation and discussion on ethical frameworks for data management and
software systems, including the IEEE and ACM codes of conduct.

Chapter 15, Ethics in Machine Learning Systems, focuses on the bias in machine
learning systems. We start by exploring sources of bias and briefly discussing
these sources. We then explore ways to spot biases, how to minimize them, and
finally, how to communicate potential biases to the users of our system.

Chapter 16, Integration of ML Systems in Ecosystems, explains how packaging the

ML systems into web services allows us to integrate them into workflows in a very
flexible way. Instead of compiling or using dynamically linked libraries, we can
deploy machine learning components that communicate over HTTP protocols using
JSON protocols. In fact, we have already seen how to use that protocol by using
the GPT-3 model that is hosted by OpenAI. In this chapter, we explore the
possibility of creating our own Docker container with a pre-trained machine
learning model, deploying it, and integrating it with other components.

Chapter 17, Summary and Where to Go Next, revisits all the best practices and
summarizes them per chapter. In addition, we also look into what the future of
machine learning and AI may bring to software engineering.

To get the most out of this book

In this book, we use Python and PyTorch, so you need to have these two installed
on your system. I used them on Windows and Linux, but they can also be used in
cloud environments such as Google Colab or GitHub Codespaces (both were
tested).
Software/hardware covered Operating system requirements
in the book

Python 3.11 Windows, Ubuntu, Debian Linux, or

Windows Subsystem for Linux (WSL)

PyTorch 2.1 Windows, Ubuntu, or Debian Linux

If you are using the digital version of this book, we advise you to type
the code yourself or access the code from the book’s GitHub repository
(a link is available in the next section). Doing so will help you avoid any
potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files for this book from GitHub at
https://github.com/PacktPublishing/Machine-Learning-Infrastructure-and-Best-
Practices-for-Software-Engineers. If there’s an update to the code, it will be
updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos
available at https://github.com/PacktPublishing/. Check them out!

Conventions used
There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names,
filenames, file extensions, pathnames, dummy URLs, user input, and Twitter
handles. Here is an example: “The model itself is created one line above, in the
model = LinearRegression() line.”

A block of code is set as follows:

def fibRec(n):
if n < 2:
 return n
else:
return fibRec(n-1) + fibRec(n-2)

Any command-line input or output is written as follows:

>python app.py

BEST PRACTICES
Appear like this.

Get in touch
Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us
at [email protected] and mention the book title in the subject of your
message.

Errata: Although we have taken every care to ensure the accuracy of our content,
mistakes do happen. If you have found a mistake in this book, we would be
grateful if you would report this to us. Please visit
www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the
internet, we would be grateful if you would provide us with the location address or
website name. Please contact us at [email protected] with a link to the
material.

If you are interested in becoming an author: If there is a topic that you have
expertise in and you are interested in either writing or contributing to a book,
please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Machine Learning Infrastructure and Best Practices for Software
Engineers, we’d love to hear your thoughts! Please click here to go straight to the
Amazon review page for this book and share your feedback.
Your review is important to us and the tech community and will help us make sure
we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books
everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that
book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your
favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters,
and great free content in your inbox daily

Follow these simple steps to get the benefits:

1. Scan the QR code or visit the link below

https://packt.link/free-ebook/978-1-83763-406-4

2. Submit your proof of purchase

3. That’s it! We’ll send your free PDF and other benefits to your email directly
Part 1:Machine Learning Landscape in
Software Engineering
Traditionally, Machine Learning (ML) was considered to be a niche domain in
software engineering. No large software systems used statistical learning in
production. This changed in the 2010s when recommendation systems started to
utilize large quantities of data – for example, to recommend movies, books, or
music. With the rise of transformer technologies, this has changed. Commonly
known products such as ChatGPT popularized these techniques and showed that
they are no longer niche products, but have entered the mainstream software
products and services. Software engineering needs to keep up and we need to
know how to create the software based on these modern machine learning
models. In this first part of the book, we look at how machine learning changes
software development and how we need to adapt to these changes.

This part has the following chapters:

Chapter 1, Machine Learning Compared to Traditional Software

Chapter 2, Elements of a Machine Learning System

Chapter 3, Data in Software Systems – Text, Images, Code, and Features

Chapter 4, Data Acquisition, Data Quality, and Noise

Chapter 5, Quantifying and Improving Data Properties

1

Machine Learning Compared to Traditional

Software
Machine learning software is a special kind of software that finds patterns in data,
learns from them, and even recreates these patterns on new data. Developing the
machine learning software is, therefore, focused on finding the right data,
matching it with the appropriate algorithm, and evaluating its performance.
Traditional software, on the contrary, is developed with the algorithm in mind.
Based on software requirements, programmers develop algorithms that solve
specific tasks and then test them. Data is secondary, although not completely
unimportant. Both types of software can co-exist in the same software system, but
the programmer must ensure compatibility between them.

In this chapter, we’ll explore where these two types of software systems are most
appropriate. We’ll learn about the software development processes that
programmers use to create both types of software. We’ll also learn about the four
classical types of machine learning software – rule-based learning, supervised
learning, unsupervised learning, and reinforcement learning. Finally, we’ll learn
about the different roles of data in traditional and machine learning software – as
input to pre-programmed algorithms in traditional software and input to training
models in machine learning software.

The best practices introduced in this chapter provide practical guidance on when
to choose each type of software and how to assess the advantages and
disadvantages of these types. By exploring a few modern examples, we’ll
understand how to create an entire software system with machine learning
algorithms at the center.

In this chapter, we’re going to cover the following main topics:

Machine learning is not a traditional software
 Probability and software – how well do they go together?

Testing and validation – the same but different

Machine learning is not traditional software

Although machine learning and artificial intelligence have been around since the
1950s, introduced by Alan Turing, they only became popular with the first MYCIN
system and our understanding of machine learning systems changed over time. It
was not until the 2010s that we started to perceive, design, and develop machine
learning in the same way as we do today (in 2023). In my view, two pivotal
moments shaped the landscape of machine learning as we see it today.

The first pivotal moment was the focus on big data in the late 2000s and early
2010s. With the introduction of smartphones, companies started to collect and
process increasingly large quantities of data, mostly about our behavior online.
One of the companies that perfected this was Google, which collected data about
our searches, online behavior, and usage of Google’s operating system, Android.
As the volume of the collected data increased (and its speed/velocity), so did its
value and the need for its veracity – the five Vs. These five Vs – volume, velocity,
value, veracity, and variety – required a new approach to working with data. The
classical approach of relational databases (SQL) was no longer sufficient.
Relational databases became too slow in handling high-velocity data streams,
which gave way to map-reduce algorithms, distributed databases, and in-memory
databases. The classical approach of relational schemas became too constraining
for the variety of data, which gave way for non-SQL databases, which stored
documents.

The second pivotal moment was the rise of modern machine learning algorithms –
deep learning. Deep learning algorithms are designed to handle unstructured data
such as text, images, or music (compared to structured data in the form of tables
and matrices). Classical machine learning algorithms, such as regression, decision
trees, or random forest, require data in a tabular form. Each row is a data point,
and each column is one characteristic of it – a feature. The classical models are
designed to handle relatively small datasets. Deep learning algorithms, on the
other hand, can handle large datasets and find more complex patterns in the data
because of the power of large neural networks and their complex architectures.

Machine learning is sometimes called statistical learning as it is based on statistical

methods. The statistical methods calculate properties of data (such as mean
values, standard deviations, and coefficients) and thus find patterns in the data.
The core characteristic of machine learning is that it uses data to find patterns,
learn from them, and then repeat these patterns on new data. We call this way of
learning patterns training, and repeating these patterns as reasoning, or in
machine learning language, predicting. The main benefits of using machine
learning software come from the fact that we do not need to design the algorithms
– we focus on the problem to be solved and the data that we use to solve the
problem. Figure 1.1 shows an example of how such a flowchart of machine
learning software can be realized.

First, we import a generic machine learning model from a library. This generic
model has all elements that are specific to it, but it is not trained to solve any
tasks. An example of such a model is a decision tree model, which is designed to
learn dependencies in data in the form of decisions (or data splits), which it uses
later for new data. To make this model somewhat useful, we need to train it. For
that, we need data, which we call the training data.

Second, we evaluate the trained model on new data, which we call the test data.
The evaluation process uses the trained model and applies it to check whether its
inferences are correct. To be precise, it checks to which degree the inferences are
correct. The training data is in the same format as the test data, but the content
of these datasets is different. No data point should be present in both.

In the third step, we use the model as part of a software system. We develop
other non-machine learning components, and we connect them to the trained
model. The entire software system usually consists of data procurement
components, real-time validation components, data cleaning components, user
interfaces, and business logic components. All these components, including the
machine learning model, provide a specific functionality for the end user. Once the
software system has been developed, it needs to be tested, which is where the
input data comes into play. The input data is something that the end user inputs
to the system, such as by filling in a form. The input data is designed in such a
way that has both the input and expected output – to test whether the software
system works correctly.

Finally, the last step is to deploy the entire system. The deployment can be very
different, but most modern machine learning systems are organized into two parts
– the onboard/edge algorithms for non-machine learning components and the user
interface, and the offboard/cloud algorithms for machine learning inferences.
Although it is possible to deploy all parts of the system on the target device (both
machine learning and non-machine learning components), complex machine
learning models require significant computational power for good performance and
seamless user experience. The principle is simple – more data/complex data
means more complex models, which means that more computational power is
needed:
 Figure 1.1 – Typical flow of machine learning software development

As shown in Figure 1.1, one of the crucial elements of the machine learning
software is the model, which is one of the generic machine learning models, such
as a neural network, that’s been trained on specific data. Such a model is used to
make predictions and inferences. In most systems, this kind of component – the
model – is often prototyped and developed in Python.

Models are trained for different datasets and, therefore, the core characteristic of
machine learning software is its dependence on that dataset. An example of such
a model is a vision system, where we train a machine learning algorithm such as a
convolutional neural network (CNN) to classify images of cats and dogs.
Since the models are trained on specific datasets, they perform best on similar
datasets when making inferences. For example, if we train a model to recognize
cats and dogs in 160 x 160-pixel grayscale images, the model can recognize cats
and dogs in such images. However, the same model will perform very poorly (if at
all!) if it needs to recognize cats and dogs in colorful images instead of grayscale
images – the accuracy of the classification will be low (close to 0).

On the other hand, when we develop and design traditional software systems, we
do not rely on data that much, as shown in Figure 1.2. This figure provides an
overview of a software development process for traditional, non-machine learning
software. Although it is depicted as a flow, it is usually an iterative process where
Steps 1 to 3 are done in cycles, each one ending with new functionality added to
the product.

The first step is developing the software system. This includes the development of
all its components – user interface, business logic (processing), handling of data,
and communication. The step does not involve much data unless the software
engineer creates data for testing purposes.

The second step is system testing, where we use input data to validate the
software system. In essence, this step is almost identical to testing machine
learning software. The input data is complemented with the expected outcome
data, which allows software testers to assess whether the software works
correctly.

The third step is to deploy the software. The deployment can be done in many
ways. However, if we consider traditional software that is similar in function to
machine learning software, it is usually simpler. It usually does not require
deployment on the cloud, just like machine learning models:
 Figure 1.2 - Typical flow of traditional software development

The main difference between traditional software and machine learning-based

software is that we need to design, develop, and test all the elements of the
traditional software. In machine learning-based software, we take an empty
model, which contains all the necessary elements, and we use the data to train it.
We do not need to develop the individual components of the machine learning
model from scratch.

One of the main parts of traditional software is the algorithm, which is developed
by software engineers from scratch, based on the requirements or user stories.
The algorithm is usually written as a sequential set of steps that are implemented
in a programming language. Naturally, all algorithms use data to operate on it, but
they do it differently than machine learning systems. They do it based on the
software engineer’s design – if x, then y or something similar.

We usually consider these traditional algorithms as deterministic, explainable, and

traceable. This means that the software engineer’s design decisions are
documented in the algorithm and the algorithm can be analyzed afterward. They
are deterministic because they are programmed based on rules; there is no
training from data or identifying patterns from data. They are explainable because
they are designed by programmers and each line of the program has a predefined
meaning. Finally, they are traceable as we can debug every step of these
programs.

However, there is a drawback – the software engineer needs to thoroughly

consider all corner cases and understand the problem very well. The data that the
software engineer uses is only to support them in analyzing the algorithm, not
training it.

An example of a system that can be implemented using both machine learning

algorithms and traditional ones is one for reading passport information. Instead of
using machine learning for image recognition, the software uses specific marks in
the passport (usually the <<< sequence of characters) to mark the beginning of the
line or the beginning of the sequence of characters denoting a surname. These
marks can be recognized quite quickly using rule-based optical character
recognition (OCR) algorithms without the need for deep learning or CNNs.

Therefore, I would like to introduce the first best practice.

BEST PRACTICE #1
Use machine learning algorithms when your problem is focused on data, not on the algorithm.

When selecting the right technology, we need to understand whether it is based

on the classical approach, where the design of the algorithm is in focus, or
whether we need to focus on handling data and finding patterns in it. It is usually
beneficial to start with the following guidelines.
If the problem requires processing large quantities of data in raw format, use the
machine learning approach. Examples of such systems are conversational bots,
image recognition tools, text processing tools, or even prediction systems.

However, if the problem requires traceability and control, use the traditional
approach. Examples of such systems are control software in cars (anti-lock
braking, engine control, and so on) and embedded systems.

If the problem requires new data to be generated based on the existing data, a
process known as data manipulation, use the machine learning approach.
Examples of such systems are image manipulation programs (DALL-E), text
generation programs, deep fake programs, and source code generation programs
(GitHub Copilot).

If the problem requires adaptation over time and optimization, use machine
learning software. Examples of such systems are power grid optimization software,
non-playable character behavior components in computer games, playlist
recommendation systems, and even GPS navigation systems in modern cars.

However, if the problem requires stability and traceability, use the traditional
approach. Examples of such systems are systems to make diagnoses and
recommendation systems in medicine, safety-critical systems in cars, planes, and
trains, and infrastructure controlling and monitoring systems.

Supervised, unsupervised, and reinforcement learning

– it is just the beginning
Now is a good time to mention that the field of machine learning is huge, and it is
organized into three main areas – supervised learning, unsupervised learning, and
reinforcement learning. Each of these areas has hundreds of different algorithms.
For example, the area of supervised learning has over 1,000 algorithms, all of
which can be automatically selected by meta-heuristic algorithms such as AutoML:
Supervised learning: This is a group of algorithms that are trained based on annotated data.
The data that’s used in these algorithms needs to have a target or a label. The label is used to
tell the algorithm which pattern to look for. For example, such a label can be cat or dog for
 each image that the supervised learning model needs to recognize. Historically, supervised
learning algorithms are the oldest ones as they come directly from statistical methods such as
linear regression and multinomial regression. Modern algorithms are advanced and include
methods such as deep learning neural networks, which can recognize objects in 3D images and
segment them accordingly. The most advanced algorithms in this area are deep learning and
multimodal models, which can process text and images at the same time.

A sub-group of supervised learning algorithms is self-supervised models,

which are often based on transformer architectures. These models do not
require labels in the data, but they use the data itself as labels. The most
prominent examples of these algorithms are translation models for natural
languages and generative models for images or texts. Such algorithms are
trained by masking words in the original texts and predicting them. For the
generative models, these algorithms are trained by masking parts of their
output to predict it.
Unsupervised learning: This is a group of models that are applied to find patterns in data
without any labels. These models are not trained, but they use statistical properties of the
input data to find patterns. Examples of such algorithms are clustering algorithms and
semantic map algorithms. The input data for these algorithms is not labeled and the goal of
applying these algorithms is to find structure in the dataset according to similarities; these
structures can then be used to add labels to this data. We encounter these algorithms daily
when we get recommendations for products to buy, books to read, music to listen to, or films
to watch.

Reinforcement learning: This is a group of models that are applied to data to solve a
particular task given a goal. For these models, we need to provide this goal in addition to the
data. It is called the reward function, and it is an expression that defines when we achieve the
goal. The model is trained based on this fitness function. Examples of such models are
algorithms that play Go, Chess, or StarCraft. These algorithms are also used to solve hard
programming problems (AlphaCode) or optimize energy consumption.

So, let me introduce the second best practice.

BEST PRACTICE #2
Before you start developing a machine learning system, do due diligence and identify the right
group of algorithms to use.
As each of these groups of models has different characteristics, solves different
problems, and requires different data, a mistake in selecting the right algorithm
can be costly. Supervised models are very good at solving problems related to
predictions and classifications. The most powerful models in this area can compete
with humans in selected areas – for example, GitHub Copilot can create programs
that can pass as human-written. Unsupervised models are very powerful if we
want to group entities and make recommendations. Finally, reinforcement learning
models are the best when we want to have continuous optimization with the need
to retrain models every time the data or the environment changes.

Although all these models are based on statistical learning, they are all
components of larger systems to make them useful. Therefore, we need to
understand how this probabilistic and statistical nature of machine learning goes
with traditional, digital software products.

An example of traditional and machine learning

software
To illustrate the difference between traditional software and machine learning
software, let’s implement the same program using these two paradigms. We’ll
implement a program that calculates a Fibonacci sequence using the traditional
approach, which we have seen a million times in computer science courses. Then,
we’ll implement the same program using machine learning models – or one model
to be exact – that is, logistic regression.

The traditional implementation is presented here. It is based on one recursive

function and a loop that tests it:

# a recursive function to calculate the fibonacci number

# this is a standard solution that is used in almost all
# of computer science examples
def fibRec(n):
if n < 2:
return n
else:
return fibRec(n-1) + fibRec(n-2)
# a short loop that uses the above function
 for i in range(23):
print(fibRec(i))

The implementation is very simple and is based on the algorithm – in our case, the
fibRec function. It is simplistic, but it has its limitations. The first one is its
recursive implementation, which costs resources. Although it can be written as an
iterative one, it still suffers from the second problem – it is focused on the
calculations and not on the data.

Now, let’s see how the machine learning implementation is done. I’ll explain this
by dividing it into two parts – data preparation and model training/inference:

#predicting fibonacci with linear regression

import pandas as pd
import numpy as np
from sklearn.linear_model import LinearRegression
# training data for the algorithm
# the first two columns are the numbers and the third column is the
result
dfTrain = pd.DataFrame([[1, 1, 2],
[2, 1, 3],
[3, 2, 5],
[5, 3, 8],
[8, 5, 13]
])
# now, let's make some predictions
# we start the sequence as a list with the first two numbers
lstSequence = [0,1]
# we add the names of the columns to make it look better
dfTrain.columns = ['first number','second number','result']

In the case of machine learning software, we prepare data to train the algorithm.
In our case, this is the dfTrain DataFrame. It is a table that contains the numbers
that the machine learning algorithm needs to find the pattern.

Please note that we prepared two datasets – dfTrain, which contains the numbers
to train the algorithm, and lstSequence, which is the sequence of Fibonacci
numbers that we’ll find later.

Now, let’s start training the algorithm:

# algorithm to train
# here, we use linear regression
model = LinearRegression()
# now, the actual process of training the model
 model.fit(dfTrain[['first number', 'second number']],
dfTrain['result'])
# printing the score of the model, i.e. how good the model is when
trained
print(model.score(dfTrain[['first number', 'second number']],
dfTrain['result']))

The magic of the entire code fragment is in the bold-faced code – the model.fit
method call. This method trains the logistic regression model based on the data
we prepared for it. The model itself is created one line above, in the model =
LinearRegression() line.

Now, we can make inferences or create new Fibonacci numbers using the
following code fragment:

# and loop through the newly predicted numbers

for k in range(23):
# the line below is where the magic happens
# it takes two numbers from the list
# formats them to an array
# and makes the prediction
# since the model returns a float,
# we need to convert it to it
intFibonacci =
int(model.predict(np.array([[lstSequence[k],lstSequence[k+1]]])))
# add this new number to the list for the next iteration
lstSequence.append(intFibonacci)
# and print it
print(intFibonacci)

This code fragment contains a similar line to the previous one – model.predict().
This line uses the previously created model to make an inference. Since the
Fibonacci sequence is recursive, we need to add the newly created number to the
list before we can make the new inference, which is done in the
lstSequence.append() line.

Now, it is very important to emphasize the difference between these two ways of
solving the same problem. The traditional implementation exposes the algorithm
used to create the numbers. We do not see the Fibonacci sequence there, but we
can see how it is calculated. The machine learning implementation exposes the
data used to create the numbers. We see the first sequence as training data, but
we never see how the model creates that sequence. We do not know whether that
model is always correct – we would need to test it against the real sequence –
simply because we do not know how the algorithm works. This takes us to the
next part, which is about just that – probabilities.

Probability and software – how well they

go together
The fundamental characteristic that makes machine learning software different
from traditional software is the fact that the core of machine learning models is
statistics. This statistical learning means that the output of the machine learning
model is a probability and, as such, it is not as clear as in traditional software
systems.

The probability, which is the result of the model, means that the answer we
receive is a probability of something. For example, if we classify an image to check
whether it contains a dog or a cat, the result of this classification is a probability –
for example, there is a 93% probability that the image contains a dog and a 7%
probability that it contains a cat. This is illustrated in Figure 1.3:

Figure 1.3 – Probabilistic nature of machine learning software

To use these probabilistic results in other parts of the software, or other systems,
the machine learning software usually uses thresholds (for example, if x<0.5) to
provide only one result. Such thresholds specify which probability is acceptable to
be able to consider the results to belong to a specific class. For our example of
image classification, this probability would be 50% – if the probability of
identifying a dog in the image is larger than 50%, then the model states that the
image contains a dog (without the probability).
Another random document with
no related content on Scribd:
 The Project Gutenberg eBook of Chambers's
Journal of Popular Literature, Science, and Art,
Fifth Series, No. 115, Vol. III, March 13, 1886
 This ebook is for the use of anyone anywhere in the United
States and most other parts of the world at no cost and with
almost no restrictions whatsoever. You may copy it, give it away
or re-use it under the terms of the Project Gutenberg License
included with this ebook or online at www.gutenberg.org. If you
are not located in the United States, you will have to check the
laws of the country where you are located before using this
eBook.

Title: Chambers's Journal of Popular Literature, Science, and

Art, Fifth Series, No. 115, Vol. III, March 13, 1886

Author: Various

Release date: May 14, 2022 [eBook #68073]

Language: English

Original publication: United Kingdom: William and Robert

Chambers, 1853

Credits: Susan Skinner, Eric Hutton and the Online Distributed

Proofreading Team at https://www.pgdp.net (This file
was produced from images generously made available
by The Internet Archive)

*** START OF THE PROJECT GUTENBERG EBOOK

CHAMBERS'S JOURNAL OF POPULAR LITERATURE, SCIENCE,
AND ART, FIFTH SERIES, NO. 115, VOL. III, MARCH 13, 1886 ***
 CHAMBERS’S JOURNAL
OF
POPULAR
LITERATURE, SCIENCE, AND
ART.
 CONTENTS
THE MONARCH OF AFRICAN MOUNTAINS.
IN ALL SHADES.
DUST AND HOUSE REFUSE: SHOWING WHAT BECOMES OF IT.
THE HAUNTED JUNGLE.
A NIGHT-RAID ON DONEGAL SMUGGLERS.
SOME FAROE LEGENDS.
THE OLD VIKING.
No. 115.—Vol. III. SATURDAY, MARCH 13, 1886. Price 1½d.
 THE MONARCH OF AFRICAN
MOUNTAINS.
To those who associate the name of the great African continent only
with visions of the steaming mangrove swamps of the west coast,
the luxuriant flower-carpets and grasses of the south, the trackless
sand-wastes of the north, and the undulating thirsty plains of ‘the
Bush,’ whose idea of Africa, indeed, may be summed up in three
words—sun, savages, and fever—to such, we say, it may be difficult
to accept the knowledge that snow-capped mountains exist in the
very heart of this dry and heat-engirdled land. But yet, there have
been for ages, strange tales of a wonderful mountain-mass in the
tropical centre, whose summit was perpetually covered with a
mysterious substance which the natives called ‘white salt.’ Now, as
perpetual snow under the equator was known only in Central
America—nowhere else do mountains in the tropics reach the snow-
line—there did exist for ages incredulity as to the existence of this
alleged African Mont Blanc or Chimborazo. The legend referring to it
must have been known to the early Portuguese travellers at least
three centuries ago, for the Portuguese were at Mombasa in the
sixteenth century, and as Mombasa is within one hundred and eighty
miles of the mountain, and is the coast-limit of the trade-route
between it and the sea, they must have heard the stories of the
native and Arab traders. Others believed this Kilima-Njaro[1] to be
merely the legendary ‘Mountains of the Moon.’
The earliest authentic record of ‘discovery’ by a European is that of
Rebmann, a German missionary, who, on the 11th of May 1848, first
sighted the wonderful snowy dome. Baron Von der Decken, another
German, actually reached Kilima-Njaro in 1861, and stayed on its
slopes for some three months. On a second visit, Von der Decken
ascended to a height of ten thousand five hundred feet, although he
did not reach the snow. He was followed, in 1871, by an English
missionary, the Rev. Charles New, who made two journeys to Chaga
—the native name for the inhabited belt between three and seven
thousand feet above the sea, stretching round the mountain—and on
the second occasion was robbed and ill-used by Mandara, a native
chief. Mr Joseph Thomson, after making the journey Through Masai-
land, of which he has published so interesting an account, arrived at
Kilima-Njaro in 1883. He journeyed nearly all round the base of the
mountain, but did not ascend more than nine thousand feet. He also
was robbed by Mandara.
It was reserved for Mr H. H. Johnston, F.R.G.S., to penetrate the
mysteries of the ‘Monarch of African Mountains,’ and to record his
experiences in a most interesting book, The Kilima-Njaro Expedition
(London: Kegan Paul). Mr Johnston’s experiences on the Congo
qualified him for African exploration; while his services to science in
other parts of the world, pointed him out as well equipped for the
search into and observation of the natural history of the locality,
selected for exploration by a joint-committee of the British
Association and the Royal Society. To solve the many interesting
problems surrounding the fauna and flora of this African alpine
region, was the task delegated to Mr Johnston. He left London in
March 1884, and in due course arrived at Zanzibar, where he was
assisted by Sir John Kirk in getting together a band of porters,
servants, and guides. After some delay at Mombasa, caused by a
sharp attack of fever, Mr Johnston plunged into the wilderness at the
head of his long band of porters, carrying loads of domestic
necessaries, provisions, water, and ‘trade’ goods. The long tramp
inland was a weary one, for it was through a hot and thirsty land,
which sorely tried the endurance of the party.
The first glimpse of Kilima-Njaro was obtained long before the party
reached its base. And here it may be proper to explain that this
name is given to the whole mountain-mass, which consists of two
huge peaks and a number of smaller ones, just below the third
parallel south of the equator. The highest of the peaks is called Kibô,
is eighteen thousand eight hundred and eighty feet above the level
of the sea, and is always covered with snow on the top, and
occasionally down to the altitude of fourteen thousand feet. This is,
so far as is at present known, the highest mountain in Africa. The
twin-peak, Kimawenzi, is sixteen thousand two hundred and fifty feet
high, and although above the snow-line, is not continuously snow-
clad. The whole mass is of volcanic origin, and the two peaks are the
craters of extinct volcanoes.
Approached from the south-east, the mountain has the appearance
of lonely isolation, and presents a truly remarkable spectacle, with its
peaks towering to the clouds and its glittering snow-caps. It is worth
while giving in Mr Johnston’s words his emotions on first gaining
sight of the goal of his desires: ‘With the falling temperature of the
small-hours, a brisk wind arose from the heated plain, and swept the
clouds from off the sky, all except the mass which obstinately clung
to Kilima-Njaro. Feverish and overtired, I could not sleep, and sat
and watched the heavens, waiting for the dawn. A hundred men
were snoring around me, and the night was anything but silent, for
the hyenas were laughing hideously in the gloom outside our circle
of expiring embers. At five o’clock I awoke my servant Virapan, and
whilst he was making my morning coffee I dropped into a doze, from
which at dawn he roused me and pointed to the horizon, where in
the north-west a strange sight was to be seen. “Laputa,” I exclaimed;
and as Virapan, though he had read Robinson Crusoe and the
Arabian Nights in his native tongue, had never heard of Gulliver’s
Travels, I proceeded to enlighten him as to the famous suspended
island of Swift’s imagining, and explained my exclamation by
pointing to the now visible Kilima-Njaro, which, with its two peaks of
Kibô and Kimawenzi, and the parent mass of mountain, rose high
above a level line of cloud, and thus completely severed in
appearance from the earth beneath, resembled so strangely the
magnetic island of Laputa.’
It was not until the thirteenth day after leaving Mombasa, that the
party entered the state of Mosi, ruled over by the chief Mandara,
already mentioned. This little kingdom is of about the same area as
London, and is on the lower slope of the mountain, between three
and four thousand feet above the sea. Splendid views are obtained
from it over the plains below, and its condition is anything but one of
savagery. The agriculture is of a high order, and the people, although
nearly naked, are both intelligent and industrious. The fields are well
intersected by artificial water-courses, led from the mountain-
streams higher up, and ‘the air is musical with the murmur of trickling
rivulets and the tinkling bells of the flocks and herds.’ Wherever the
ground is not in cultivation, it is covered with brilliantly coloured wild
flowers of numberless known and unknown species; the hum of bees
is suggestive of endless stores of honey; and the flow of milk is
guaranteed by the innumerable herds of mild-eyed kine cropping the
rich pasture.
Finding that the feuds between the Mosi people and the other
mountain tribes were a bar to his progress through Mandara’s
country, Mr Johnston withdrew, and negotiated treaties of peace and
commerce with one of the rival potentates whose territory extended
nearer the summit. Before doing this, however, he had to retire to a
place called Taveita, through which he had passed on his way to
Mandara’s. Of this place he says: ‘From the day of my first arrival up
to the time of my final departure, it seemed to me one of the loveliest
spots on the earth’s surface.’
Taveita is the sort of trade centre of the district, and is ruled over by
a senate of notables, called the ‘Wazēē,’ or elders, who preserve law
and order, and arbitrate in disputes between the resident natives and
the nomadic traders. Its population is about six thousand.
From Taveita, Mr Johnston negotiated with the chief of Maranū state
rather larger than Middlesex, on the south-eastern flank of the
mountain. After many preliminaries and much exchanging of
presents, he was at length admitted into this kingdom, and had
positively to crawl into it through the defensive stockades, which it
seems the custom in this country for the separate peoples to erect
around their domains. Between the kingdom of Maranū and the
summit of Kibô, there lay no opposing tribe, so that, having obtained
guides, Mr Johnston was, after a little delay, enabled to continue his
journey to the snow.
The route crossed a fine river, and lay at first through a smiling and
fertile country, with signs of cultivation and flourishing banana-groves
up to an altitude of five thousand five hundred feet. Shortly after that,
cultivation ceased, and a heathy district was reached, with grassy
knolls and numerous small streams of running water. The ascent
was very gradual, and the first night was spent in camp at six
thousand five hundred feet. Leaving this, a dense forest was
reached at seven thousand feet; then a district of uplands thickly
covered with moss and ferns, studded with short gnarled trees, and
teeming with begonias and sweet-scented flowering shrubs, but with
few signs of animal life. At nine thousand feet, the region was clear
of forests, and merely covered with grass; but higher up, the
woodland began again, and water became very abundant. The third
camp was formed at ten thousand feet, and here the party
encountered a terrific thunderstorm and rainfall. It was succeeded by
a fair and serene morning, leaving the two snow-peaks in full view
against a cloudless blue sky. At this point Mr Johnston resided nearly
a month, actively prosecuting his collecting and observing, and
preparing for the final ascent. Then, one day, with three followers
only, he started for great Kibô.
For some two thousand feet higher, vegetation is abundant; and
even at twelve thousand six hundred feet the party struck a pretty
little stream, on the banks of which were patches of level greensward
and abundance of gay flowers, while the spoor of buffaloes was also
observed. Strange sessile thistles, five feet in circumference, were
noticed; and an extraordinary lobelia, between three and four feet in
height, with bright-blue blossoms, as also other remarkable plants.
Bees and wasps were still to be seen at this high altitude, and bright
little sunbirds darting about. But beyond thirteen thousand feet,
vegetation was seen only in dwarfed patches, and the ground
became covered with boulders, lying in confused masses, with
occasional huge slabs of rock, singularly marked like tortoise-shells.
At thirteen thousand six hundred feet, the last resident bird was
noticed—a species of stonechat—although high-soaring kites and
great-billed ravens were seen even higher up. At fourteen thousand
one hundred and seventeen feet, the Zanzibari followers were
thoroughly done up, and began to show unmistakable signs of fear
of the ‘bogey’ of the mountain, so they were left to prepare a
sleeping-place for the night, while Mr Johnston continued the ascent
alone.
At fifteen thousand one hundred and fifty feet he reached the central
connecting ridge of Kilima-Njaro, and could see part of both sides.
The ‘Monarch,’ however, was veiled in clouds. What followed cannot
better be given than in the adventurer’s own words: ‘At length—and
it was so sudden and so fleeting, that I had no time to fully take in
the majesty of the snowy dome of Kibô—the clouds parted, and I
looked on a blaze of snow so blinding white under the brief flicker of
sunlight, that I could see little detail. Since sunrise that morning I had
caught no glimpse of Kibô, and now it was suddenly presented to me
with unusual and startling nearness.... Knowing now the direction of
my goal, I rose from the clammy stones, and clutching my sketch-
book with benumbed hands, began once more to ascend westwards.
Seeing but a few yards in front of me, choked with mist, I made but
slow progress; nevertheless, I continually mounted along a gently
sloping, hummocky ridge, where the spaces in between the masses
of rock were filled with fine yellowish sand. The slabs of rock were so
slippery with the drizzling mist, that I very often nearly lost my
footing, and I thought with a shudder what a sprained ankle would
mean here.
‘At length, after a rather steeper ascent than usual up the now
smoother and sharper ridge, I suddenly encountered snow lying at
my very feet, and nearly plunged headlong into a great rift filled with
snow, that here seemed to cut across the ridge and interrupt it. The
dense mist cleared a little in a partial manner, and I then saw to my
left the black rock sloping gently to an awful gulf of snow, so vast and
deep that its limits were concealed by fog. Above me a line of snow
was just discernible, and altogether the prospect was such a gloomy
one, with its all-surrounding curtain of sombre cloud, and its
uninhabited wastes of snow and rock, that my heart sank within me
at my loneliness.... Turning momentarily northwards, I rounded the
rift of snow, and once more dragged myself, now breathless and
panting, and with aching limbs, along the slippery ridge of bare rock,
which went ever mounting upwards.... The feeling that overcame me
when I sat and gasped for breath on the wet and slippery rocks at
this great height, was one of overwhelming isolation. I felt as if I
should never more regain the force to move, and must remain and
die amid this horrid solitude of stones and snow. Then I took some
brandy-and-water from my flask, and a little courage came back to
me. I was miserably cold, the driving mist having wetted me to the
skin. Yet the temperature recorded here was above the freezing-
point, being thirty-five degrees Fahrenheit.... The mercury rose to
183.8. This observation, when properly computed, and with the
correction added for the temperature of the intermediate air, gives a
height of sixteen thousand three hundred and fifteen feet as the
highest point I attained on Kilima-Njaro.’
When he returned to the camping-place, Mr Johnston found that his
three followers had deserted him, being thoroughly terrified, and
certain that the white man had perished on the lonely heights. With
much difficulty he made his way to the station on the lower ground,
where the great body of his attendants had remained; and in due
course the whole party arrived safely again at Taveita. From there a
new route was taken, by way of Lake Jipé, to the coast at Pangani,
where the followers were paid off. An English mission afforded Mr
Johnston shelter until he could get a passage on an Arab dau to
Zanzibar, where he caught the mail-steamer; and in little more than
six weeks after getting his last glimpse of the snow-peaks of Kilima-
Njaro, from the shores of Lake Jipé, the gallant explorer was in
London once more.
Although attaining the highest altitude yet reached by man in Africa,
Mr Johnston did not complete the conquest of Kilima-Njaro. But he
reached within two thousand feet of the summit; and having shown
the way, it will be odd if some of the adventurous spirits among
alpine climbers do not essay the task of peering into the hidden
depths of the crater of Kibô. Be this as it may, the expedition has
resulted in the acquisition of a vast amount of valuable information
about the geography, the fauna, and flora of this strange district,
where in two days you can ascend from equatorial heat to arctic
cold. Even in the plains, the temperature is, for six months in the
year, quite bearable, and in some parts delightful. The extreme
fertility of the mountain slopes, the abundance of game, the stores of
ivory to be obtained from the vast herds of elephants, the rare and
beautiful skins—in short, all the known riches of animal and
vegetable production, and the supposed existence of mineral
deposits, such as copper and nitrate of soda, point to this district as
destined to play an important part in the future of Africa.
 IN ALL SHADES.
CHAPTER XIII.
‘Father, father,’ Dr Whitaker whispered in a low voice, ‘let us go
aside a little—down into my cabin or somewhere—away from this
crowd here. I am so glad, so happy to be back with you again; so
delighted to be home once more, dear, dear father. But don’t you
see, everybody is looking at us and observing us!’
The old mulatto glanced around him with an oily glance of profound
self-satisfaction. Yes, undoubtedly; he was the exact centre of an
admiring audience. It was just such a house as he loved to play to.
He turned once more to his trembling son, whose sturdy knees were
almost giving way feebly beneath him, and redoubled the ardour of
his paternal demonstrativeness. ‘My son, my son, my own dear boy!’
he said once more; and then, stepping back two paces and opening
his arms effusively, he ran forward quickly with short mincing steps,
and pressed the astonished doctor with profound warmth to his
swelling bosom. There was an expansiveness and a gushing
effusion about the action which made the spectators titter audibly;
and the titter cut the poor young mulatto keenly to the heart with a
sense of his utter helplessness and ridiculousness in this absurd
situation. He wondered to himself when the humiliating scene would
ever be finished. But the old man was not satisfied yet. Releasing his
son once more from his fat grasp, he placed his two big hands
akimbo on his hips, puckered up his eyebrows as if searching for
some possible flaw in a horse or in a woman’s figure—he was a
noted connoisseur in either—and held his head pushed jauntily
forward, staring once more at his son with his small pig’s eyes from
top to toe. At last, satisfied apparently with his close scrutiny, and
prepared to acknowledge that it was all very good, he seized the
young doctor quickly by the shoulders, and kissing him with a loud
smack on either cheek, proceeded to slobber him piecemeal all over
the face, exactly like a nine-months’-old baby. Dr Whitaker’s cheeks
tingled and burned, so that even through that dusky skin, Edward,
who stood a little distance off, commiserating him, could see the hot
blood rushing to his face by the deepened and darkened colour in
the very centre.
Presently, old Bobby seemed to be sufficiently sated with this
particular form of theatrical entertainment, and turned round
pleasantly to the remainder of the company. ‘My son,’ he said, not
without a real touch of heart-felt, paternal pride, as he glanced
towards the gentlemanly looking and well-dressed young doctor,
‘your fellow-passengers! Introduce me! Which is de son of my ole
and valued friend, de Honourable James Hawtorn, of Wagwater?’
Dr Whitaker, glad to divert attention from himself on any excuse,
waved his hand quietly towards Edward.
‘How do you do, Mr Whitaker?’ Edward said, in as low and quiet a
tone as possible, anxious as he was to disappoint the little gaping
crowd of amused spectators. ‘We have all derived a great deal of
pleasure from your son’s society on our way across. His music has
been the staple entertainment of the whole voyage. We have
appreciated it immensely.’
But old Bobby was not to be put off with private conversation aside in
a gentle undertone. He was accustomed to living his life in public,
and he wasn’t going to be balked of his wonted entertainment. ‘Yes,
Mr Hawtorn,’ he answered in a loud voice, ‘you are right, sah. De
taste for music an’ de taste for beauty in de ladies are two tastes dat
are seldom wantin’ to de sons or de grandsons of Africa, however far
removed from de original negro.’ (As he spoke, he glanced back with
a touch of contempt and an infinite superiority of manner at the pure-
blooded blacks, who were now busily engaged in picking up
portmanteaus from the deck, and squabbling with one another as to
which was to carry the buckras’ luggage. Your mulatto, however
dark, always in a good-humoured, tolerant way, utterly despises his
coal-black brethren.) ‘Bote dose tastes are highly developed in my
own pusson. Bote no doubt my son, Wilberforce Clarkson Whitaker,
is liable to inherit from his fader’s family. In de exercise of de second,
I cannot fail to perceive dat dis lady beside you must be Mrs
Hawtorn. Sah’—with a sidelong leer of his fat eyes—‘I congratulate
you mos’ sincerely on your own taste in female beauty. A very nice,
fresh-lookin’ young lady, Mrs Hawtorn.’
Marian’s face grew fiery red; and Edward hardly knew whether to
laugh off the awkward compliment, or to draw himself up and stroll
away, as though the conversation had reached its natural ending.
‘And de odder young lady,’ Bobby went on, quite unconscious of the
effect he had produced—‘de odder young lady? Your sister, now, or
Mrs Hawtorn’s?’
‘This is Miss Dupuy of Orange Grove,’ Edward answered
hesitatingly; for he hardly knew what remark old Bobby might next
venture upon. And indeed, as a matter of fact, the old mulatto’s
conversation, even in the presence of ladies, was not at all times
restrained by all those artificial rules of decorum imposed on most of
us by what appeared to him a ridiculously strait-laced and puritanical
white conventionality.
But Edward’s answer seemed to have an extraordinary effect in
sobering and toning down the old man’s exuberant volubility; he
pulled off his hat with a respectful bow, and said in a lower and more
polite voice: ‘I have de honour of knowing Miss Dupuy’s fader; I am
proud to make Miss Dupuy’s acquaintance.’
‘Here, Bobby!’ the captain called out from a little forward—‘you come
here, say. The first-officer wants to introduce you’—with a wink at
Edward—‘to His Excellency the Peruvian ambassador.—Look here,
Mr Hawthorn; don’t you let Bobby talk too long to your ladies, sir. He
sometimes blurts out something, you know, that ladies ain’t exactly
accustomed to. We seafaring men are a bit rough on occasion
ourselves, certainly; but we know how to behave for all that before
the women.—Bobby, don’t; you’d better be careful.’
‘Thank you,’ Edward said, and again felt his heart smitten with a sort
of remorse for poor Dr Whitaker. That quick, sensitive, enthusiastic
young man to be tied down for life to such a father! It was too
terrible. In fact, it was a tragedy.
‘Splendid take-down for that stuck-up, young brown doctor,’ the
English officer exclaimed aside in a whisper to Edward. ‘Shake a
little of the confounded conceit out of him, I should say. He wanted
taking down a peg.—Screaming farce, isn’t he, the old father?’
‘I never saw a more pitiable or pitiful scene in my whole life,’ Edward
answered earnestly. ‘Poor fellow, I’m profoundly sorry for him; he
looks absolutely broken-hearted.’
The young officer gazed at him in mute astonishment. ‘Can’t see a
joke, that fellow Hawthorn,’ he thought to himself. ‘Had all the fun
worked out of him, I suppose, over there at Cambridge. Awful prig!
Quite devoid of the sense of humour. Sorry for his poor wife; she’ll
have a dull life of it.—Never saw such an amusing old fool in all my
days as that ridiculous, fat old nigger fellow!’
Meanwhile, James Hawthorn had been standing on the wharf,
waiting for the first crush of negroes and hangers-on to work itself
off, and looking for an easy opportunity to come aboard in order to
meet his son and daughter. By-and-by the crush subsided, and the
old man stepped on to the gangway and made his way down upon
the deck.
In a moment, Edward was wringing his hand fervently, and father
and son had exchanged one single kiss of recognition in that half-
shamefaced, hasty fashion in which men of our race usually get
through that very un-English ceremony of greeting.
‘Father, father,’ Edward said, ‘I am so thankful to see you once more;
so anxious to see my dear mother.’
There were tears standing in both their eyes as his father answered:
‘My boy, my boy! I’ve denied myself this pleasure for years; and now
—now it’s come, it’s almost too much for me.’
There was a moment’s pause, and then Mr Hawthorn turned to
Marian. ‘My daughter,’ he said, kissing her with a fatherly kiss, ‘we
know you, and love you already, from Edward’s letters; and we’ll do
our best, as far as we can, to make you happy.’
There was another pause, and then the father said again: ‘You didn’t
get my telegram, Edward?’
‘Yes, father, I got it; but not till we were on the very point of starting.
The steamer was actually under weigh, and we couldn’t have
stopped even if we had wished to. There was nothing for it but to
come on as we were, in spite of it.’
‘Oh, Mr Hawthorn, there’s papa!’ Nora cried excitedly. ‘There he is,
coming down the gangway.’ And as she spoke, Mr Dupuy’s portly
form was seen advancing towards them with slow deliberateness.
For a second, he gazed about him curiously, looking for Nora; then,
as he saw her, he walked over towards her in his leisurely, dawdling,
West Indian fashion. Nora darted forward and flung her arms
impulsively around him. ‘So you’ve come, Nora,’ the old gentleman
said quietly, disembarrassing himself with elephantine gracefulness
from her close embrace—‘so you’ve come, after all, in spite of my
telegram!—How was this, my dear? How was this, tell me?’
‘Yes, papa,’ Nora answered, a little abashed at his serene manner.
‘The telegram was too late—it was thrown on board after we’d
started. But we’ve got out all safe, you see.—And Marian—you know
—Marian Ord—Mrs Hawthorn that is now—she’s taken great care of
me; and, except for the hurricane, we’ve had such a delightful
voyage!’
Mr Dupuy drew himself up to his stateliest eminence and looked
straight across at Marian Hawthorn with stiff politeness. ‘I didn’t know
it was to Mrs Hawthorn, I’m sure,’ he said, ‘that I was to be indebted
for your safe arrival here in Trinidad. It was very good of Mrs
Hawthorn, I don’t doubt, to bring you out to us and act as your
chaperon. I am much obliged to Mrs Hawthorn for her kind attention
and care of you on the voyage. I must thank Mrs Hawthorn very
sincerely for the trouble she may have been put to on your account.
—Good-morning, Mrs Hawthorn!—Good-morning, Mr Hawthorn!
Your son, I suppose? Ah, so I imagined.—Good-morning, good-
morning.’ He raised his hat with formal courtesy to Marian, and
bowed slightly to the son and father. Then he drew Nora’s arm
carefully in his, and was just about to walk her immediately off the
steamer, when Nora burst from him in the utmost amazement and
rushed up to kiss Marian. ‘Papa,’ she cried, ‘I don’t think you