Professional Documents
Culture Documents
The Ai Revolution in Networking Cybersecurity and Emerging Technologies Omar Santos Full Chapter
The Ai Revolution in Networking Cybersecurity and Emerging Technologies Omar Santos Full Chapter
The authors and publisher have taken care in the preparation of this
book, but make no expressed or implied warranty of any kind and
assume no responsibility for errors or omissions. No liability is
assumed for incidental or consequential damages in connection with
or arising out of the use of the information or programs contained
herein.
For information about buying this title in bulk quantities, or for
special sales opportunities (which may include electronic versions;
custom cover designs; and content particular to your business,
training goals, marketing focus, or branding interests), please
contact our corporate sales department at
[email protected] or (800) 382-3419.
For government sales inquiries, please contact
[email protected].
Hoboken, NJ
ISBN-13: 978-0-13-829369-7
ISBN-10: 0-13-829369-4
$PrintCode
Editor-in-Chief
Mark Taub
Executive Editor
James Manly
Managing Editor
Sandra Schroeder
Development Editor
Christopher A. Cleveland
Production Editor
Mary Roth
Copy Editor
Jill Hobbs
Technical Editor
Petar Radanliev
Editorial Assistant
Cindy Teeters
Cover Designer
Chuti Prasertsith
Composition
codeMantra
Indexer
Erika Millen
Proofreader
Jennifer Hinchliffe
I would like to dedicate this book to my lovely wife, Jeannette, and
my
two beautiful children, Hannah and Derek, who have inspired and
supported me
throughout the development of this book.
—Omar Santos
—Samer Salam
—Hazim Dahir
Contents
Preface
Significant Milestones in AI
Development (This Book Is Already
Obsolete)
It is crucial to be humble and recognize a unique paradox inherent to this
field: The accelerating pace of AI development might render any effort to
encapsulate its current state obsolete almost as soon as it is
documented. In this regard, you could argue that this book—or, really,
any book about technology—captures a snapshot of a moment already
surpassed by the exponential rate of progress.
Each day, AI research generates new insights and unveils improved
algorithms, models, and implementations. These developments are not
only related to the breakthroughs that make news headlines, or what you
hear in podcasts and YouTube videos, but also thrive in the form of
myriad incremental advancements that might seem small on their own
but collectively represent a significant transformation. The landscape of
AI we explore today might differ from the one that exists tomorrow.
However, instead of viewing this inability to keep up as a shortcoming,
consider it a testament to the true potential and pace of the AI field.
Don’t think about this book as just a static record of AI’s current state,
but rather view it as a compass, pointing toward the broader impact. It’s
designed to provide a framework, a lens through which to make sense of
this ongoing revolution, and to help navigate the future developments
that, at this moment, we can only begin to imagine.
The AI field has already seen an incredible number of important
milestones, many of which have led to the advacements we’re seeing
today. Figure 1-1 provides a timeline of the most popular historical
milestones of AI.
Figure 1-1
Historical AI Development Milestones
Note
Feature engineering is the process in which the data scientist specifies
which aspects of the data the model should pay attention to.
Tip
In the context of AI, parameters are the internal variables that the
model learns through training. They are the part of the model that is
learned from historical training data and enable the model to make
predictions or decisions. In a simple linear regression model, the
parameters are the slope and the y-intercept. In a deep learning
model such as a neural network, the parameters are the weights and
biases in the network. These parameters are initially set to random
values and then iteratively adjusted based on the feedback signal
(loss or error) that the model gets as it trains on the data.
In the case of LLMs like GPT, the parameters are the weights in the
numerous layers of the Transformer architecture that the model uses.
As an example, the legacy GPT-3 has 175 billion parameters, which
means the model has an equal number of weights that it can adjust
to learn from the data it’s trained on. GPT-4 is speculated to have 1.76
trillion parameters, though some sources suggest it is a combination
of different models (the exact details have not been disclosed by
OpenAI).
Their total number of parameters allows these models to capture and
represent very complex patterns and relationships in the data. In turn,
that is part of what enables them to generate such remarkably
human-like text.
Tip
Training a model from scratch can be computationally expensive and
time-consuming. Transfer learning can significantly speed up this
training process. When you have a small dataset, training a model
from scratch may result in overfitting. Transfer learning helps in such
cases by leveraging a pretrained model. A pretrained model has
generalized features that can enhance its performance on the new
task, even if the new task is significantly different from the original
task. Transfer learning has seen successful applications in multiple
domains, including NLP, computer vision, and reinforcement learning.
Figure 1-2
Mechanics of Variable Transformation
Tip
Retrieval augmented generation (RAG) is an NLP technique that
combines the strengths of extractive retrieval and sequence-to-
sequence generative models to produce more informative and
contextually relevant responses. In a typical RAG setup, an initial
retrieval model scans a large corpus of documents to find relevant
passages based on the query, and these retrieved passages are then
provided as an additional context to a sequence-to-sequence model
that generates the final response. This process enables the model to
access external knowledge effectively, enriching its generated
responses or answers with information that may not be present in the
initial training data. In a typical RAG implementation, vector
databases such as Chroma DB and Pinecone are used to store the
vectorized representation of the data.
The following article explains how you can use RAG in AI
implementations: https://community.cisco.com/t5/security-
blogs/generative-ai-retrieval-augmented-generation-rag-and-
langchain/ba-p/4933714.
Each of these models has specific use cases where they shine. The best
model to use often depends on the specific data and task at hand.
Tip
Neural networks can be used for both supervised and unsupervised
learning tasks, as well as a combination of the two, known as semi-
supervised learning. The categorization depends on the specific
problem being solved and the type of data available. In supervised
tasks like classification or regression, neural networks are trained
using labeled data. The network learns to map inputs to the correct
outputs (labels) through back-propagation and iterative optimization
of a loss function. Examples include image classification, sentiment
analysis, and time-series prediction.
In unsupervised tasks, neural networks are trained without labels to
find underlying patterns or representations in the data. Techniques
like autoencoders and generative adversarial networks (GANs) are
examples of neural networks used in unsupervised learning. They are
often used for tasks like anomaly detection, dimensionality reduction,
and data generation.
Some neural networks leverage both labeled and unlabeled data to
improve learning performance. This is especially useful when
acquiring a fully labeled dataset will be expensive or time-consuming.
Although they do not strictly fall under the categories of supervised or
unsupervised, neural networks can also be used in reinforcement
learning; in this application, they are trained to make a sequence of
decisions to maximize some notion of cumulative reward.
All of the models in Table 1-3 are quite data-hungry and computationally
intensive models; however, they have achieved state-of-the-art results in
their respective domains. They are also characterized by the ability to
learn features automatically from raw data—a key advantage over many
traditional machine learning models.
Earlier, I mentioned Transformer-based models. What are those models?
Well, let’s ask ChatGPT using GPT-4:
“explain Transformer-based AI models like I am 5 years old.”
Imagine you’re trying to tell a story with your toys. But there’s a catch:
Each toy can remember only one thing at a time. It gets difficult to tell a
really good story, right? Because the toys don’t know what the other
toys are doing, they can’t react properly or help each other out.
Now, imagine that each of your toys could talk to all the other toys at
the same time, and remember what the other toys are doing. This way,
they can work together and understand the story much better, because
they’re all sharing what they know. That would make your story much
more fun and exciting!
and
“Tupatunan tenakomejona.”
“... he claps their ends together in quick succession, and by so doing produces
rhythmic clanks to suit the style of any dance or the time of any song.”