% ! TeX program = lualatex
\documentclass[a4paper,11pt]{article} 
% packages
\usepackage{censor}
\StopCensoring
\usepackage{fontspec}
\setmainfont{EB Garamond}
% for tironian et fallback
% % \directlua{luaotfload.add_fallback
% % ("emojifallback",
% %      {"Noto Serif:mode=harf"}
% % )}
% % \setmainfont{EB Garamond}[RawFeature={fallback=emojifallback}]

\setmonofont[Scale=MatchLowercase]{Deja Vu Sans Mono}
\usepackage[a4paper,left=2cm,right=2cm,top=\dimexpr15mm+1.5\baselineskip,bottom=2cm]{geometry}
\setlength{\parindent}{0pt}

\usepackage{fancyhdr}       % Headers and footers 
\fancyhead[R]{\normalfont \leftmark}
\fancyhead[L]{}
\pagestyle{fancy}

\usepackage{microtype}      % Slightly tweak font spacing for aesthetics
\usepackage[english]{babel} % Language hyphenation and typographical rules
\usepackage{xcolor}
\definecolor{linkblue}{RGB}{0, 64, 128}
\usepackage[final, colorlinks = false, urlcolor = linkblue]{hyperref} 
% \newcommand{\secref}[1]{\textbf{§~\nameref{#1}}}
\newcommand{\secref}[1]{\textbf{§\ref{#1}~\nameref{#1}}}

\usepackage{array}
\usepackage{booktabs}
\usepackage{mathtools}
\usepackage{changepage}     % adjust margins on the fly

\usepackage{minted}
\usemintedstyle{algol_nu}

\usepackage{pgfplots}
\pgfplotsset{width=\textwidth,compat=1.9}

\usepackage{caption}
\newenvironment{code}{\captionsetup{type=listing}}{}
\captionsetup[listing]{skip=0pt}
\setlength{\abovecaptionskip}{5pt}
\setlength{\belowcaptionskip}{5pt}

\usepackage[yyyymmdd]{datetime}
\renewcommand{\dateseparator}{--}

\usepackage{enumitem}

\usepackage{titlesec}

\author{Andrew Hayes}

\begin{document}
\begin{titlepage}
    \begin{center}
        \hrule
        \vspace*{0.6cm}
        \Huge \textsc{ct420}
        \vspace*{0.6cm}
        \hrule
        \LARGE
       \vspace{0.5cm}
       Real-Time Systems
       \vspace{0.5cm}
       \hrule

       \vfill

       \hrule
        \begin{minipage}{0.495\textwidth} 
            \vspace{0.4em}
            \raggedright
            \normalsize 
            \begin{tabular}{@{}l l}
                Name: & Andrew Hayes \\
                Student ID: & 21321503 \\
                E-mail: & \href{mailto://a.hayes18@universityofgalway.ie}{a.hayes18@universityofgalway.ie} \\
            \end{tabular}
        \end{minipage}
        \begin{minipage}{0.495\textwidth} 
            \raggedleft
            \vspace*{0.8cm}
            \Large
            \today
            \vspace*{0.6cm}
        \end{minipage}
        \medskip\hrule 
    \end{center}
\end{titlepage}

\pagenumbering{roman}
\newpage
\tableofcontents
\newpage
\setcounter{page}{1}
\pagenumbering{arabic}

\section{Introduction}
\subsection{Lecturer Contact Information}
\begin{itemize}
    \item   Name: Dr. Michael Schukat.
    \item   E-mail: \href{mailto://michael.schukat@universityofgalway.ie}{michael.schukat@universityofgalway.ie}.
    \item   Office: CSB-3002.
\end{itemize}

\begin{itemize}
    \item   Name: Dr. Jawad Manzoor.
    \item   E-mail: \href{jawad.manzoor@universityofgalway.ie}{jawad.manzoor@universityofgalway.ie}.
    \item   Office: CSB-3012.
\end{itemize}

\subsection{Assessment}
\begin{itemize}
    \item   2 hours of face-to-face \& virtual labs per week from Week 03.
    \item   30\% Continuous Assessment:
            \begin{itemize}
                \item   2 assignments, 10\% each.
                \item   2 in-class quizzes between Week 07 \& Week 12, worth 5\%.
            \end{itemize}
\end{itemize}

\subsection{Introduction to Real-Time Systems}
A system is said to be \textbf{real-time} if the total correctness of an operation depends not only upon its logical correctness but also upon the time in which it is performed.
Contrast functional requirements (logical correctness) versus non-functional requirements (time constraints).
There are two main categorisation factors:
\begin{itemize}
    \item   \textbf{Criticality:}
            \begin{itemize}
                \item   \textbf{Hard RTS:} deadlines (responsiveness) is critical.
                        Failure to meet these have severe to catastrophic consequences (e.g., injury, damage, death).
                \item   \textbf{Soft RTS:} deadlines are less critical, in many cases significant tolerance can be permitted.
            \end{itemize}

    \item   \textbf{Speed}
            \begin{itemize}
                \item   \textbf{Fast RTS:} responses in microseconds to hundreds of microseconds.
                \item   \textbf{Slow RTS:} responses in the range of seconds to days.
            \end{itemize}
\end{itemize}

A \textbf{safety-critical system (SCS)} or life-critical system is a system whose failure or malfunction may result in death or serious injury to people, loss of equipment / property or severe damage, \& environmental harm.

\section{The Essence of Time: From Measurement to Navigation \& Beyond}
\textbf{Time} is the continued sequence of existence \& events that occurs in an apparently irreversible succession from past, through the present, into the future.
Methods of temporal measurement, or chronometry, take two distinct forms:
\begin{itemize}
    \item   The \textbf{calendar}, a mathematical tool for organising intervals of term;
    \item   The \textbf{clock}, a physical mechanism that counts the passage of time.
\end{itemize}

Global (maritime) exploration requires exact maritime navigation, i.e., longitude \& latitude calculation.
\textbf{Latitude} (north-south) orientation is straightforward; \textbf{longitude} (east-west orientation) requires a robust (maritime) clock.
\\\\
\textbf{Ground-based navigation systems} like LORAN (LOng RAnge Navigation) were developed in the 1940s and were in use until recently, and required fixed terrestrial longwave radio transmitters, and receivers on-board of ships \& planes.
They are also referred to as hyperbolic navigation or multilateration.
The principles of ground-based navigation systems is as follows:
\begin{enumerate}
    \item   A \textbf{master} with a known location broadcasts a radio pulse.
    \item   Multiple \textbf{slave} stations with a known distance from the master send their own pulse, upon receiving the master pulse.
    \item   A \textbf{receiver} receives master \& slave pulses and measures the delay between them.
    \item   This allows the receiver to deduce the distance to each of the stations, providing a fix.
\end{enumerate}

NEED TO FINISH

\section{Time Synchronisation in Distributed Systems}
A \textbf{distributed system (DS)} is a type of networked system wherein multiple computers (nodes) work together to perform a task.
Such systems may or may not be connected to the Internet.
Time \& synchronisation are important issues here: think of error logs in distributed systems -- how can error events recorded in different computers be correlated with each other if there is no common time base?
The problem is that GNSS-based time synchronisation may or may not be available, as GPS signals are absorbed or weakened by building structures.
There is no other time reference such systems can rely on because in such a distributed system there are just a series of imperfect computer clocks.
\\\\
In distributed systems, all the different nodes are supposed to have the same notion of time, but quartz oscillators oscillate at slightly different frequencies.
Hence, clocks tick at different rates (called \textit{clock skew}), resulting in an increasing gap in perceived time.
The difference between two clocks at a given pot is called \textit{clock offset}.
The \textbf{clock synchronisation problem} aims to minimise the clock skew and subsequently the offset between two or more clocks.
A clock can show a positive or negative offset with regard to a reference clock (e.g., UTC), and will need to be resynchronised periodically.
One cannot just set the clock to the ``correct'' time: jumps, particularly backwards, can confuse software and operating systems.
Instead, we aim for gradual compensation by correcting the skew: if a clock runs too fast, make it run slower until correct and if a clock runs too slow, make it run faster until correct.
\\\\
Synchronisation can take place in different forms:
\begin{itemize}
    \item   Based on \textbf{physical} clocks: absolute to each other by synchronising to an accurate time source (e.g., UTC), absolute to each other by synchronising to locally agreed time (i.e., no link to a global time reference), where the term \textit{absolute} means that the differences in timestamps are proper time intervals.

    \item   Based on \textbf{logical} clocks (i.e., clocks are more like counters): timestamps may be ordered but with no notion of measurable time intervals.
\end{itemize}

In either case, the DS endpoints synchronise using a shared network.
For physical clock synchronisation, network latencies must be considered as packets traverse from a sending node to a receiving node.
In a \textbf{perfect network}, messages \textit{always} arrive, with a propagation delay of \textit{exactly} $d$;
the sender sends time $T$ in a message, the receiver sets its clock to $T + d$, and synchronisation is exact.
\\\\
In a \textbf{deterministic network}, messages arrive with a propagation delay $0 < d \leq D$;
the sender sends time $T$ in a message, the receiver sets its clock to $T + \frac{D}{2}$, and therefore the synchronisation error is at most $\frac{D}{2}$.
\textbf{Deterministic communication} is the ability of a network to guarantee that a message will be transmitted in a specified, predictable period of time.  

\subsection{Synchronisation in the Real World}
Most off-the-shelf networks are \textit{asynchronous}, that is, data is transmitted intermittently on a best-effort basis.
They are designed for flexibility, not determinism, and as a result, propagation delays are arbitrary and sometimes even unsymmetric (i.e., upstream \& downstream latencies are different).
Therefore, synchronisation algorithms are needed to accommodate these limitations.

\subsubsection{Cristian's Algorithm}
\textbf{Cristian's algorithm} attempts to compensate for symmetric network delays:
\begin{enumerate}
    \item   The client remembers the local time $T_0$ just before sending a request.
    \item   The server receives the request, determines $T_S$, and sends it as a reply.
    \item   When the client receives the reply, it notes the local arrival time $T_1$.
    \item   The correct time is then approximately $(T_S + \frac{(T_1 - T_0)}{2} )$.
\end{enumerate}

The algorithm assumes symmetric network latency.
If the server is synced to UTC< all clients will follow UTC.
Limitations of Cristian's algorithm include:
\begin{itemize}
    \item   Assumes a symmetric network latency;
    \item   Assumes that timestamps can be taken as the packet hits the wire / arrives at the client;
    \item   Assumes that $T_S$ is right in the middle of the server process; for example, consider the server process being pre-empted just before it sends the response back to the client, which will corrupt the synchronisation of the client.
\end{itemize}

\subsubsection{Berkeley Algorithm}
In the \textbf{Berkeley algorithm}, there is no accurate time server: instead, a set of client clocks is synchronised to their average time.
The assumption is that offsets / skews of all clocks follow some symmetric distribution (e.g., a normal distribution) with some clocks going faster and others slower, and therefore a mean value close to 0.
\begin{enumerate}
    \item   One node is designated to be the \textbf{master node} $M$.
    \item   The master node periodically queries all other clients for their local time.
    \item   Each client returns a timestamp or their clock offset to the master.
    \item   Cristian's algorithm is used to determine and compensate for RTTs, which can be different for each client.
    \item   Using these, the master computes the average time (thereby ignoring outliers), calculates the difference to all timestamps it has received, and sends an adjustment to each client.
            Again, each computer gradually adjusts its local clock.
\end{enumerate}

Client clocks are adjusted to run faster or slower, to be synced to an overall agreed system time.
The client networks is an intranet, i.e., an isolated system.
Therefore, the Berkeley algorithm is an \textbf{internal clock synchronisation algorithm}.
The Berkeley algorithm was implemented in the TEMPO time synchronisation protocol, which was part of the Berkelely UNIX 4.3BSD system.

\subsection{Logical Clocks}
\textbf{Logical clocks} are another concept linked to internal clock synchronisation.
Logical clocks only care about their internal consistency, but not about absolute (UTC) time;
subsequently, they do not need clock synchronisation and take into account the order in which events occur rather than the time at which they occurred.
In practice, if clients or processes only care that event $a$ happens before event $b$, but don't care about the exact time difference, they can make use of a logical clock.
\\\\
We can define the \textbf{happens-before relation} $a \rightarrow b$:
\begin{itemize}
    \item   If events $a$ and $b$ are within the same process, then $a \rightarrow b$ if $a$ occurs with an earlier local timestamp: \textbf{process order}.
    \item   If $a$ is the event of a message being sent by one process, and $b$ is the event of the message being received by another process, then $a \rightarrow b$: \textbf{causal order}.
    \item   We also have \textbf{transitivity:} if $a \rightarrow b$ and $b \rightarrow c$, then $a \rightarrow c$.
\end{itemize}
Note that this only provides a \textit{partial order}:
if two events $a$ and $b$ happen in different processes that do not exchange messages (not even indirectly), then neither $a \rightarrow b$ nor $b \rightarrow a$ is true.
In this situation, we say that $a$ and $b$ are \textbf{concurrent} and write $a \sim b$, i.e., nothing can be said about when the events happened or which event happened first.
\\\\
Happens-before can be implemented using the \textbf{Lamport scheme:}
\begin{enumerate}
    \item   Each process $P_i$ has a logical clock $L_i$, where $L_i$ can be simply an integer variable initialised to 0.
    \item   $L_i$ is incremented on every local event $e$; we write $L_i(e)$ or $L(e)$ as the timestamp of $e$.
    \item   When $P_i$ sends a message, it increments $L_i$ and copies its content into the packet.
    \item   When $P_i$ receives a message from $P_k$, it extracts $L_k$ and sets $L := \text{max}(L_i, L_k)$ and then increments $L_i$.
\end{enumerate}

This guarantees that if $a \rightarrow b$, then $L_i(a) < L_k(b)$, but nothing else.
\\\\
The primary limitation of Lamport clocks is that they do not capture \textbf{causality}.
Lamport's logical clocks lead to a situation where all events in a distributed system are ordered, so that if an event $a$ (linked to $P_i$) ``happened before''  event $b$ (linked to $P_k$), i.e., $a \rightarrow b$, then $a$will allso be positioned in that ordering before $b$ such that $L_i(a) < L_k(b)$ or simply $L(a) < L(b)$;
however, nothing can be said about the relationship between two events $a$ \& $b$ by merely comparing their time values $L_i(a)$ and $L_k(b$.): we can't tell if $a \rightarrow b$ or $b \rightarrow a$ or $a \sim b$ unless they occur in the same process.

\subsubsection{Vector Clocks}
In practice, causality is captured by means of \textbf{vector clocks}:
\begin{enumerate}
    \item   There is an ordered list of logical clocks, with one per process.
            Each process $P_i$ maintains a vector $\vec{V_i}$, initially containing all zeroes.
            Each index $k$ of a vector clock $\vec{V_i}[k]$ represents the number of events that process $P_i$ knows have occurred in process $P_k$.
            $\vec{V_i}[i]$  is the count of events that have occurred locally at process $P_i$, while $\vec{V_i}[k]$ (for $k\neq i$) is the count of events in process $P_k$ that $P_i$ is aware of.

    \item   On a local event $e$, $P_i$ increments its own clock component $\vec{V_i}[i]$.
            If the event is ``message send'', a new $\vec{V_i}$ is copied into the packet, so that on message sends the current vector state is included in the message.

    \item   If $P_i$ receives a message from $P_m$, then each index $\vec{V_i}[k]$ where $k \neq i$ is set to $\text{max}(\vec{V_m}[k], \vec{V_i}[k])$, and $\vec{V_i}[i]$ is incremented.
\end{enumerate}

Intuitively, $\vec{V_i}[k]$is the number of events in process $P_k$ that have been observed by $P_i$.

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/vector_clocks_example.png}
    \caption{Vector clocks example}
\end{figure}

In the above example, when process $P_2$ receives message $m_1$, it merges the entries from $P_1$'s clock by choosing the maximum value of each position.
Similarly, when $P_3$ receives $m_2$, it merges in $P_2$'s clock, thus incorporating the changes from $P_1$ that $P_2$ already saw. 
Vector clocks explicitly track the transitive causal order: $f$'s timestamp captures the history of $a$, $b$, $c$, \& $d$.
\\\\
To use vector clocks for ordering, we can compare them piecewise:
\begin{itemize}
    \item   We say $\vec{V_i} = \vec{V_j}$ if and only if $\vec{V_i}[k] = \vec{V_j}[k] \forall k$.
    \item   We say $\vec{V_i} \leq \vec{V_j}$ if and only if $\vec{V_i}[k] \leq \vec{V_j}[k] \forall k$.
    \item   We say $\vec{V_i} < \vec{V_j}$ if and only if $\vec{V_i} \leq \vec{V_j}$ and $\vec{V_i} \neq \vec{V_j}$.
    \item   We say $\vec{V_i} \sim \vec{V_j}$ otherwise, e.g., $\vec{V_i} = [2,0,0]$ and $\vec{V_j} = [0,0,1]$.
\end{itemize}

For any two event timestamps $T(a)$ \& $T(b)$:
\begin{itemize}
    \item   if $a \rightarrow b$, then $T(a) < T(b)$; \textbf{and}
    \item   if $T(a) < T(b)$, then $a \rightarrow b$.
\end{itemize}

Hence, we can use timestamps to determine if there is a causal ordering between any two events.

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/vector_vs_lamport.png}
    \caption{Lamport clocks versus vector clocks}
\end{figure}

\section{The NTP Protocol}
The options for computer clocks are essentially as follows:
\begin{itemize}
    \item   \textbf{Option A:} stick to crystals.
            \begin{itemize}
                \item   Costly precision manufacturing.
                \item   Works indoors.
                \item   Temperature Compensated Crystal Oscillator (TCXO).
                \item   Oven Controlled Crystal Oscillator (OCXO).
            \end{itemize}

    \item   \textbf{Option B:} buy an atomic clock (\$50,000 -- \$100,000) or a GNSS receiver (based on an atomic clock, but doesn't work indoors), or a time signal radio receiver if you are based in central Europe.

    \item   \textbf{Option C:} use software-based approaches to discipline cheap crystal clocks.
            Less quality, but useful for certain applications, and works indoors.
\end{itemize}

\textbf{Distributed master clocks} provide a time reference to hosts that are inter-connected via a network.
The underlying time-synchronisation protocols combine aspects of Cristian's algorithm (i.e., RTD calculation) and Berkeley's algorithm (i.e., combining multiple reference time sources).
Good time synchronisation requires:
\begin{itemize}
    \item   Good time references: easily done with GPS, atomic clocks, etc.
    \item   Predictable / symmetric / deterministic network latencies: doable in LAN setups, but not guaranteed in Internet data communication.
\end{itemize}

There are two main distributed master clock protocols:
\begin{itemize}
    \item   \textbf{Network Time Protocol (NTP):} defined in RFC 5905, originally used in the Unix-based NTP daemon, one of the first Internet protocols that ever evolved.
    \item   \textbf{Precision Time Protocol (PTP):} designed for managed networks, e.g., LAN.
\end{itemize}

\begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{./images/ntpvsptp.png}
    \caption{NTP \& PTP characteristics}
\end{figure}

In \textbf{uni-directional synchronisation} a reference clock sends a timestamp to a host via a network.
The host then uses the timestamp to set its local clock.
Useful when message latencies are minor relative to the synchronisation levels required.
\\\\
In \textbf{round-trip synchronisation (RTS)} a host sends a request message, receives a reference clock response message with known (local) submission \& arrival times, allowing for the calculation of the round-trip delay (1) and the host clock error (2), i.e., the phase offset.
Variations of RTS form the basis for NTP \& PTP.

\begin{align}
    \delta = (T_{i+3} - T_i) - (T_{i+2} - T_{i+1})
\end{align}
\begin{align}
    \theta = (T_{i+1}-T_i) + \frac{(T_{i+2}-T_{i+3})}{2}
\end{align}

\begin{figure}[H]
    \centering
    \includegraphics[width=0.4\textwidth]{./images/rts.png}
    \caption{ RTS example }
\end{figure}

\subsection{NTP}
The NTP architecture, protocol, \& algorithms have evolved over the last 40 years to the latest NTP Version 4.
NTP is the standard Internet protocol for time synchronisation \& co-ordinated UTC time distribution.
It is a fault-tolerant protocol, as it automatically selects the best of several available time sources to synchronise with.
It is highly scalable, as the nodes form a hierarchical structure with reference clock(s) at the top:
\begin{itemize}
    \item   Stratum 0: Time Reference Source (e.g., GPS, TAI atomic clocks, DCF 77).
    \item   Stratum 1: Primary Time Server.
\end{itemize}

NTP applies some general aforementioned principles such as avoiding setting clocks backwards and avoiding large step changes;
the required change (positive or negative) is amortised over a series of short intervals (e.g., over multiple ticks).
\\\\
NTP is the longest-running and continuously operating Internet protocol (since around 1979).
Government agencies in many other countries and on all continents (including Antarctica) operate public NTP primary servers.
National \& regional service providers operate public NTP secondary servers synchronised to the primary servers.
Many government agencies, private \& public institutions including universities, broadcasters, financial institutions, \& corporations operate their own NTP networks. 
\begin{figure}[H]
    \centering
    \includegraphics[width=0.5\textwidth]{./images/ntphierarchy.png}
    \caption{ NTP hierarchy }
\end{figure}

In \textbf{client/server mode}, UDP is used for data transfer (no TCP), i.e., NTP over UDP on UDP port 123.
There is also the optional use of broadcasting or multicasting (not covered here).
Several packet exchanges between the NTP client and the Stratum server take place to determine the client offset:
\begin{enumerate}
    \item   The client sends a packet with \textbf{originate timestamp} $A$.
    \item   The server receives the packet and returns a response containing the originate timestamp $A$ as well as the \textbf{receive timestamp} $B$ and the \textbf{transmit timestamp} $C$.
    \item   The client receives this packet and processes $A$, $B$, $C$, as well as the packet arrival time $D$ of the received packet; it then determines the offset and the round-trip delay (RTD).
\end{enumerate}

\begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{./images/ntpoperation.png}
    \caption{ NTP operation }
\end{figure}

The above example is of a symmetric network with a 15ms delay each way; the client's clock lags 5ms behind the server's clock:
\begin{align*}
    \text{RTD}      = (D-A)-(C-B) = 32-2 =& 30\text{ms} \\
    \text{Offset}   = \frac{((B-A)+(C-D))}{2} = \frac{(20 + (-10))}{2} =& 5\text{ms}
\end{align*}

\begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{./images/networkdelayasymmetry.png}
    \caption{ Network delay asymmetry }
\end{figure}

In the above example, the client's clock still lags 5ms behind the servers clock, but there is an asymmetric network latency: 10ms versus 20ms:
\begin{align*}
    \text{RTD}      = (D-A)-(C-B) = 32-2                                =& 30\text{ms} \\
    \text{Offset}   = \frac{((B-A)+(C-D))}{2} = \frac{(15 + (-15))}{2}  =& 0\text{ms}
\end{align*}

Typical NTP performance for various set-ups is as follows:
\begin{itemize}
    \item   Small LAN: \sim10 microseconds best possible case on a 2-node LAN, \sim220 microseconds on a real-world small LAN.
    \item   Typical large-building LAN: \sim2ms.
    \item   Internet with a few hops: 10–20ms.
    \item   Long distance and/or slow or busy link: 100ms–1s.
\end{itemize}

Accuracy is further degraded on networks with asymmetric traffic delays.
\\\\
The \textbf{NTP time format} has a reference scale of UTC.
Time parameters are 64 bits long:
\begin{itemize}
    \item   Seconds since January 1, 1900 (32 bits, unsigned).
    \item   Fraction of second (32 bits, unsigned).
\end{itemize}
The NTP time format has a dynamic range of 136+ years, with rollover in 2036.
Its resolution is 2\textsuperscript{-32} seconds \sim232 picoseconds.

\subsubsection{NTP Protocol Header}
\begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{./images/ntpheader.png}
    \caption{ NTP protocol header }
\end{figure}
\begin{itemize}
    \item   \verb|LI| (\textbf{Leap Indicator}) 2-bit:
            \begin{itemize}
                \item   \verb|0|: no warning;
                \item   \verb|1|: last minute of the day has 61 seconds;
                \item   \verb|2|: last minute of the day has 59 seconds.
            \end{itemize}


    \item   \verb|VN| (\textbf{Version Number}) 2-bit: currently 4. 

    \item   \verb|Mode|: 3-bit integer, including:
            \begin{itemize}
                \item   \verb|3|: client;
                \item   \verb|4|: server.
            \end{itemize}

    \item   \verb|Stratum|: 8-bit integer for Stratum server hierarchy level, including:
            \begin{itemize}
                \item   \verb|1|: primary server (i.e., stratum 1);
                \item   \verb|2|–\verb|15|: secondary server.
            \end{itemize}

    \item   \verb|Poll|: 8-bit signed integer representing the maximum interval between successive message, in $\log_2$ seconds.
            This field indicates the interval at which the client will poll the NTP server for time updates.
            The client dynamically adjusts this interval based on its clock's stability \& the network conditions to balance accuracy \& network load.

    \item   \verb|Precision|: 8-bit signed integer representing the resolution of the system clock (the tick increment) in $\log_2$ seconds;
            e.g., a value of -18 corresponds to a resolution of about one microsecond (2\textsuperscript{-18}) seconds.

    \item   \verb|Root Delay|: round-trip packet delay from a client to a stratum 1 server.
            It gives a crude estimate of the worst-case time transfer error between a client and a stratum 1 server due to network asymmetry, i.e., if all of the round-trip delay was in one direction and none in the other direction.
\end{itemize}

For a single client clock, the \textbf{dispersion} is a measure of how much the client's clock might drift during a synchronisation cycle:
\begin{align*}
    \text{Dispersion} = \textit{DR} \times (D-A) + \textit{TS}
\end{align*}
where $(D-A)$ is the duration of a synchronisation cycle, with $A$ being the first timestamp and $D$ being the last timestamp, \textit{DR} being the local clock skew (i.e., the deviation of actual clock tick frequency to nominal clock tick frequency), and \textit{TS} being the timestamping errors due to the finite resolution of the clock and delays in reading the clock when fetching a timestamp.
\\\\
The \textbf{root dispersion} of a client clock is the combined dispersions of all stratum servers along the path to a Stratum 1 server.
\\\\
The \textbf{root distance} is the sum of root dispersion and half the root delay.
It provides a comprehensive measure of the maximum error in time synchronisation as the total worst case timing error accumulated between the Stratum 1 server and the client.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{./images/rootdispersion.png}
    \caption{ Example root dispersion \& root delay }
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{./images/annotatedntpheader.png}
    \caption{ Annotated NTP protocol header }
\end{figure}

A \textbf{reference ID (refid)} is a 32-bit code (4 ASCII bytes) identifying the particular server or reference clock, e.g.,:
\begin{itemize}
    \item   \verb|GPS|: Global Positioning System;
    \item   \verb|GAL|: Galileo Positioning System;
    \item   \verb|PPS|: Generic Pulse-Per-Second;
    \item   \verb|DCF|: LF Radio DCF77 Mainflingen, DE 77.5 kHz;
    \item   \verb|WWV|: HF Radio WWV Fort Collins, Colorado;
    \item   \verb|GOOG|: unofficial Google refid used by Google NTP servers as \verb|time4.google.com|.
\end{itemize}

\begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{./images/ntparchitecuraloverview.png}
    \caption{ NTP architectural overview }
\end{figure}

An NTP client synchronises with multiple stratum servers.
It uses a range of algorithms to deal with variable \& asymmetric non-deterministic network delays and to determine its most likely offset, thereby running a series of processes:
\begin{itemize}
    \item   The peer process runs when a packet is received;
    \item   The poll process sends packets at intervals determined by the clock discipline process \& remote server;
    \item   The system process runs when a new update is received;
    \item   The clock discipline process implements clock time adjustments;
    \item   The clock adjust process implements periodic clock frequency (VFO) adjustments.
\end{itemize}

For each stratum there is a \textbf{poll process} that sends NTP packets at intervals ranging from 8 seconds to 36 hours.
The corresponding \textbf{peer processes} receive NTP packets and, after performing some packet sanity tests, $T1$ – $T4$ are determined / extracted.
The NTP daemon calculates offset \& delay as seen before.
The time series of offset \& delay values calculated by multiple peer processes are processed by a sequence of algorithms, thus eliminating servers with long RTD or servers that show ``strange'' offsets which, for example, are often the result of network asymmetries.

\subsubsection{Mitigation Algorithms}
The \textbf{clock filter algorithm} uses a sliding window of eight samples for each stratum server and picks out the sample with the least expected error, i.e., it chooses the sample with the minimum RTD.
It is effective at removing spikes resulting from intermittent network congestions.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{./images/clockfilteralg.png}
    \caption{ Clock filter algorithm before and after }
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{./images/clockfilteralgmotivation.png}
    \caption{ 
        The wedge scattergram plots sample points of offset versus delay (RTD) collected over a 24-hour period by a client clock communicating with a single stratum server.
        For this experiment, the client clock is externally synced to the stratum server, so the offset should be zero;
        however, as the (network) round trip delay increases, the offset variability increases, resulting in increasingly larger offset errors.
        Therefore, the best samples are those at the lowest delay.
        This is taken into account by the clock filter algorithm.
    }
\end{figure}

The \textbf{intersection algorithm} selects a subset of peers (i.e., stratum servers) and identifies truechimers \& truetickers based on the intersection of confidence (offset) intervals (i.e., min/max offsets of a clock over $x$ readings determines its invterval).

\begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{./images/intersectional.png}
    \caption{ Plot range of offsets calculated by each peer with 1, 2, \& 3 overlapping}
\end{figure}

\textbf{Marzullo's algorithm} is an agreement protocol for estimating accurate time from a number of noisy time sources by intersecting offset intervals; if some intervals don't intersect, it considers the intersection of the majority of intervals.
It eliminates false tickers.
\\\\
The \textbf{clock cluster algorithm} processes the truechimers returned by the clock intersection algorithm.
It produces a list of survivors by eliminating truechimers that have a comparably large root delay \& root dispersion.
Finally, the \textbf{clock combining algorithm} averages the time offsets of survivors using their root dispersions as a weight, i.e., survivors with a small root dispersion have a higher weight.
\\\\
The \textbf{combining algorithm} provides a final offset, so the client clock can be adjusted.
The UNIX Clock Model provides the kernel variable \verb|tickadj| which amortises the required change gradually by making adjustments every tick e.g., every 10 milliseconds.

\section{Soft RTS}
A \textbf{soft RTS} is a RTS in which performance is degraded but not destroyed by failure to meet response time constraints, e.g., multimedia systems.

\subsection{Quality of Service (QoS)}
\textbf{Perceived QoS} reflects the subjective evaluation of service quality from the end user's perspective and overall satisfaction.
It encompasses factors like application responsiveness, ease of use, consistency, reliability, and the overall user interface delay.
It is challenging to measure accurately as it involves subjective experiences \& user feedback, often gathered through surveys, user studies, or feedback mechanisms.
\\\\
\textbf{Intrinsic QoS} refers to the technical \& measurable characteristics of a network or service that directly affects its performance \& reliability.
Intrinsic QoS parameters include bandwidth, latency, packet loss rate, jitter, \& throughput.
Intrinsic QoS measures are typically quantifiable and can be objectively measured using various tools \& monitoring techniques.

\subsubsection{QoS Metrics}
\textbf{Latency} is the time it takes to send a packet from point $A$ (say, the client) to point $B$ (say, the server).
It is physically limited by how fast signals can travel in wires or the open air.
Latency depends on the physical, real-world distance between $A$ and $B$.
Typical latencies are conceptually small, between roughly 10 and 200 milliseconds.
High latency can result in delays between user actions and system responses, leading to sluggish or unresponsive behaviour in real-time applications.
\\\\
\textbf{Jitter} is the inconsistency or fluctuations in the arrival time of data packets at the receiver.
It can be caused by various factors, such as network congestion, packet loss, routing changes, etc.
It can have significant implications for real-time communication applications, particularly voice \& video sharing.
Inconsistent packet arrival times can lead to disruptions, distortion, and out-of-sync audio or video playback.
\\\\
\textbf{Bandwidth} measures the amount of data that is able to pass through a network at a given time.
It is measured in bits per second.
Real-time applications with high-bandwidth requirements, such as high-definition video streaming \& VOIP may experience performance issues if the available bandwidth is insufficient to accommodate the data transmission demands.

\subsection{Real-Time Multimedia Technologies}
\subsubsection{Transport Layer for Real-Time Media}
\textbf{TCP} provides reliability, ordered delivery, \& congestion control.
Retransmissions can lead to high delay and cause delay jitter.
TCP is not suitable for real-time systems and does not support mulitcast.
\\\\
\textbf{UDP} has no built-in reliability or congestion control, and no defined technique for synchronising.
It has low latency and minimal overhead (no handshake, no retransmissions).
A feedback channel must be defined for quality control.

\subsubsection{Service Requirements for Real-Time Flows (Voice / Video)}
\begin{itemize}
    \item   \textbf{Sequencing:} the process of maintaining the correct order of data packets during transmission and ensuring that they are re-assembled correctly at the receiver's end.
    \item   \textbf{Synchronisation:} ensures that different types of data streams (such as audio \& video) are aligned in time during playback.
    \item   \textbf{Payload identification:} different media types (MPEG1, MPEG2, H.261) may require different handling in terms of decoding or processing.
    \item   \textbf{Frame indication:} specifying which packets belong to the same frame or video sequence and helps with decoding \& rendering video frames accurately.
\end{itemize}

\subsubsection{RTP}
\textbf{Real-time Transport Protocol (RTP)} provides end-to-end transport functions suitable for real-time audio/video applications over multicast \& unicast network services.
It works in user space over UDP.
The working model is as follows:
\begin{itemize}
    \item   The multimedia application generates multiple streams (audio, video, etc.) that are fed into the RTP library.
    \item   The library multiplexes the streams \& encodes them in RTP packets which are fed to a UDP socket.
\end{itemize}

\textbf{Secure RTP (SRTP)} is used by applications including WhatsApp, Zoom, Skype, etc. for transporting voice \& video streams.

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/RTP.png}
    \caption{ RTP protocol}
\end{figure}

RTP services include:
\begin{itemize}
    \item   Payload type identification: determination of media encoding.
    \item   Synchronisation source identification: assigned to each distinct media source (such as a microphone or a camera).
            Enables synchronisation of multiple streams coming from the same source (e.g., lip-syncing audio \& video).
    \item   Sequence numbering: a counter is used which is incremented on each RTP packet send and is used to detect lost packets.
    \item   Time-stamping: time monitoring, synchronisation, jitter calculation.
    \item   RTP issues: no QoS guarantees, no guarantee of packet delivery.
\end{itemize}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/RTPpheader.png}
    \caption{ RTP header}
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/RTPdatadelirvery.png}
    \caption{ RTP data delivery}
\end{figure}

\textbf{RTP Control Protocol (RTCP)} is a companion protocol to RTP that is used periodically to transmit control packets to participants in a streaming multimedia session.
It gathers statistics on the media connection, including bytes sent, packets sent, lost packets, jitter, feedback, \& round-trip delay. 
It provides feedback on the quality of the service being provided by RTP but does not actually transport any data.
The application may use this information to increase the quality of service, perhaps by limiting flow or using a different codec.

\subsubsection{Mouth-to-Ear M2E Delay}
\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/m2e.png}
    \caption{ M2E delay }
\end{figure}

M2E delay consists of:
\begin{itemize}
    \item   Sender:
            \begin{itemize}
                \item   Packetisation delay.
                \item   Encoding delays.
                \item   OS/Application/Driver software.
                \item   MAC delays.
            \end{itemize}

    \item   Network:
            \begin{itemize}
                \item   Propagation delay.
                \item   Queuing delays.
                \item   Processing/serialisation delays in routers.
            \end{itemize}

    \item   Receiver:
            \begin{itemize}
                \item   NIC delays.
                \item   OS/Application/Driver software.
                \item   Jitter buffer delays.
                \item   Decoding delays.
            \end{itemize}
\end{itemize}

VOIP QoS strategies include:
\begin{itemize}
    \item   Sender-based:
            \begin{itemize}
                \item RTCP feedback with adaptive codeces: if loss/delay excessive, switch to lower-bandwidth code or implement Forward Error Correction (FEC) strategy.
            \end{itemize}

    \item   Network-based:
            \begin{itemize}
                \item   Prioritising delay-sensitive traffic flows.
            \end{itemize}

    \item   Receiver-based:
            \begin{itemize}
                \item   Buffering strategies: the human ear is not sensitive to short-term variations;
                    the buffer ``absorbs'' variation in network queueing delays and the voice can be reconstructed using RTP timestamps, but this adds to overall M2E delay.
                \item   Packet Loss Concealment (PLC) and FEC.
            \end{itemize}
\end{itemize}

\subsection{Packet Loss Strategies}
The use of UDP limits delays but can lead to packet loss;
to mitigate this, compensation strategies can be employed at the sender or receiver:
\begin{itemize}
    \item   Forward Error Correction (FEC): a form of information redundancy.
    \item   Packet Loss Concealment (PLC): silence (simplest), repetition of last packet, or interpolation.
\end{itemize}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/jitterbufferstrat.png}
    \caption{ Receiver-based packet loss strategies: jitter buffer strategies }
\end{figure}

The buffer playout delay adds to the M2E delay.
In the above figure, packet 8 arrives too late for the playout; we could drop the packet, or increase the buffer size in response to increasing delays.
With a fixed buffer, the limitations are that if it is too large, it extends overall delay, and if it is too small, there will be additional late packet losses due to late arrival.
An adaptive buffer size can adapt to network conditions:
\begin{itemize}
    \item   \textbf{Per Talkspurt (PT):} operates by elongating/shortening inter talkspurt silence periods; less noticeable.
    \item   \textbf{Per Packet Scaling (PPS):} speed up/slow down speech, like in Skype or Zoom.
\end{itemize}

\subsubsection{Network-Based Strategies}
In a LAN environment: switched LANs are typically QoS enabled, and fast ethernet links are rarely congested.
In a WAN environment, we could increase bandwidth (which is costly and mostly a temporary solution) or employ a reservation policy, traffic categorisation, \& prioritisation which requires an admission control policy.

\subsection{Cloud Gaming}
In \textbf{cloud gaming}, the game is installed and executed on a powerful remote server located in the cloud.
The game rendering (processing graphics, physics, \& game logic) is done on this server.
Once the game is rendered on the server, the video frames \& audio are compressed and streamed to the player's device via the internet.
The player's inputs, such as controller buttons, mouse movements, or keyboard presses are sent back to the could server over the Internet in real-time.
It is commonly used in Massively Multiplayer Online Games (MMOG), and the server must deal with multiple players.
Cloud gaming solves several issues of the traditional approach:
\begin{itemize}
    \item   Tedious installation \& patching process.
    \item   Extensive hardware \& GPUs.
\end{itemize}

It is an RTC application, so we need to minimise delays (lag) and need time synchronisation to measure delays.
Simultaneity is also required: all players need to see the same game representation.

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/cloudgaming.png}
    \caption{Client/server architecture of GamingAnywhere}
\end{figure}

\subsubsection{Response Delay}
\begin{itemize}
    \item   \textbf{Response Delay (RD)} is the time difference between a user submitting a command and the corresponding in-game action appearing on the screen.
    \item   \textbf{Processing Delay (PD)} is the time required to receive / process a player's command and encode / transmit the corresponding frame.

    \item   \textbf{Playout Delay (OD)} is the time required for the client to received, decode, and render a frame on the display.

    \item   \textbf{Network Delay (ND)} is the round-trip delay.
\end{itemize}

\begin{align*}
    \text{Response Delay (RD)} = \text{PD} + \text{OD} + \text{ND}
\end{align*}

\subsection{WebRTC}
\textbf{Web Real-Time Communications (WebRTC)} is a free, open-source project that provides web browsers \& mobile applications with real-time communication (RTC) via simple APIs.
It allows audio \& video communication to work inside web pages via direct peer-to-peer communication using JavaScript \& HTML, that is, eliminating the need to install plugins or download native apps.
It uses DTLS for encryption \& SRTP for secure media streams.
It works on major web browsers (Chrome, Firefox, Safari, Edge) and mobile platforms.
Use cases include the web versions of Google Meet, Discord, Facebook Messenger, peer-to-peer file sharing, etc.

\subsubsection{WebRTC Working Model}
\begin{enumerate}
    \item   \textbf{Media capture:} accesses camera \& microphone using browser APIs.
    \item   \textbf{Connection management:} exchanges session control messages (via WebSockets, SIP, or custom methods) to set up connections.
    \item   \textbf{Data transmission:} WebRTC uses the Interactive Connectivity Establishment (ICE) techniques to overcome the complexities of real-world networking like NAT:
            \begin{itemize}
                \item   STUN: finds public IP addresses.
                \item   TURN: relays media if direct connections fail.
            \end{itemize}

        RTP/SRTP streams media, ensuring low-latency delivery.

    \item   \textbf{Peer discovery:} in P2P communication, the peers need to be able to identify or locate each other over the wire.
            Peer discovery mechanisms are not defined by WebRTC, although the process can be as simple as sharing a URL that peers can use to communicate.

    \item   \textbf{ICE techniques:} ICE will first try to make a connection using the host address obtained from a device's operating system.
            If the network is unsuccessful, ICE will obtain an external address using the STUN server.
            If that fails, the traffic is routed via a TURN relay server.
\end{enumerate}

\textbf{Session Traversal Utilities for NAT (STUN)} is a protocol that is used to discover public addresses that determines any restrictions in your router that would prevent a direct connection to a peer.
Clients receive their public addresses as requested from STUN servers.
\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/stun.png}
    \caption{STUN}
\end{figure}

\textbf{Traversal Using Relays around NAT (TURN)} bypasses the symmetric NAT restriction by opening a connection with a TURN server and relaying all information through that server.
A connection is required with a TURN server which will tell all the peers to send packets to the server which will then be forwarded to the requester.
\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/turn.png}
    \caption{TURN}
\end{figure}

\subsection{DASH}
\textbf{DASH} stands for Dynamic Adaptive Streaming over HTTP:
\begin{itemize}
    \item   \textbf{Dynamic:}  reacts to changing scenarios.
    \item   \textbf{Adaptive:} has media represented in different it rates / codecs.
    \item   \textbf{Streaming:} not strictly real-time communication, but timelines are still important.
    \item   \textbf{HTTP:} web model, HTTP good for firewalls etc.
\end{itemize}

The DASH working model is as follows:
\begin{itemize}
    \item   \textbf{Encoding \& Segmentation:} the original video/audio content is divided into small segments (usually 2-10 seconds each).
            Each segment is encoded at multiple bitrates \& resolutions for adaptability.
            A manifest file is created, containing metadata about segments, their URLs, codecs, \& timing.

    \item   \textbf{Delivery:} segments and the MPD file are uploaded to HTTP servers or CDNs.
            The encoded video segments are pushed out to client devices over the Internet.

    \item   \textbf{Playback:} the client downloads the MPD file to understand the available content \& quality options.
        It chooses appropriate ``representation'' based on network conditions, device capabilities, \& user preferences, decodes the chunks, and plays back the video.

    \item   \textbf{Quality adjustment:} the player continuously downloads \& plays segments, adjusting the quality as network conditions change.
\end{itemize}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/dash.png}
    \caption{DASH}
\end{figure}


\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/dashdatamodel.png}
    \caption{DASH data model}
\end{figure}

\section{Emerging Protocols}
\begin{table}[H]
    \centering
    \renewcommand{\arraystretch}{1.3}
    \begin{tabular}{|p{4cm}|p{11cm}|}
        \hline
        \textbf{QoS Attribute} & \textbf{Definition} \\
        \hline
        Accuracy & The error rate produced by the service \\
        \hline
        Accessibility & The degree to which the service is capable of serving a Web service request \\
        \hline
        Capacity & The limit of concurrent requests for guaranteed performance \\
        \hline
        Response Time & The time to complete a Web service request (from the perspective of a client) \\
        \hline
        Throughput & The number of Web service requests served at a given time period \\
        \hline
        Availability & The probability that the service can respond to the consumer requests \\
        \hline
        MTTR & Mean Time To Repair \\
        \hline
        Interoperability & The ease with which a consumer application or agent interoperates with a service \\
        \hline
        Robustness & The degree to which a service can function correctly in the presence of invalid, incomplete or conflicting inputs \\
        \hline
        Authentication & A measure of how the service authenticates principals who can access the service \& data \\
        \hline
        Confidentiality & A measure of how the service protects the data, so that only authorized principals can access or modify the data \\
        \hline
    \end{tabular}
    \caption{Quality of Service (QoS) Attributes and Their Definitions}
\end{table}

\textbf{Website speed}, or \textit{website performance}, refers to how quickly a browser is able to load fully-functional web pages for a given site.
Site speed is important for user experience, SEO, \& bounce rate.
Faster sites yield more page views, higher engagement, \& higher conversion rates.

\subsection{How to achieve speed?}
The speed limit of information is $c$, the speed of light.
The two main ways of increasing speed are:
\begin{itemize}
    \item   Improving the communication link; \&
    \item   Improving the protocols: app protocols provide useful user-level functions, transport protocols provide guarantees to apps, network protocols provide best-effort global packet delivery, \& link protocols provide best-effort local packet delivery.
\end{itemize}

\subsection{Evolution of the Web}
There is a paradigm shift occurring in the way in which the Internet is accessed:
there is a transformation from native applications to web applications.
Traditional applications are getting migrated to the cloud, while web applications \& services are becoming extremely complex.
QoS is very important in web applications.
The complexity of web content over the years has necessitated updates to the HyperText Transfer Protocol (HTTP).

\subsection{HTTP}
\subsubsection{HTTP/0.9}
In 1991, the first documented official version of HTTP was written as a plain document, less than 700 words in length, named \textbf{HTTP/0.9}
This version only supported the \verb|GET| method, allowing clients to retrieve HTML documents from a server, but not supporting any other file formats or information upload.

\subsubsection{HTTP/1.0}
In May 1996, RFC 1945 was published as a final \textbf{HTTP/1.0}.
In HTTP/1.0, each request/response pair requires opening a new connection.
New features of HTTP/1.0 included:
\begin{itemize}
    \item   \textbf{Header:} the HTTP header was introduced, thereby allowing the transmission of metadata that made the protocol flexible \& extensible;
    \item   \textbf{Status code:} HTTP responses now contained a status code, thereby enabling the receiver to check the request processing status;
    \item   \textbf{Content-type:} HTTP could transmit other document types other than a plain HTML file;
    \item   \textbf{New methods:} two new methods, \verb|POST| \& \verb|HEAD|, were added in addition to \verb|GET|.
\end{itemize}

Establishing a TCP connection using a three-way handshake is expensive;
2 RTTs (Round-Trip Time) are required between the client and the server to establish a connection.

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/tcpthreesome.png}
    \caption{The TCP 3-way handshake is expensive}
\end{figure}

\subsubsection{HTTP/1.1}
In January 1997, RFC 2068 was officially released as \textbf{HTTP/1.1}.
Its most relevant enhancements were:
\begin{itemize}
    \item   \textbf{Persistent connections:} in HTTP/1.1, it is possible to execute several requests using a single connection, and thus amortise the cost of the initial connection establishment and the cost of slow starts across multiple requests.
    \item   \textbf{New methods:} besides the already-available methods of HTTP/1.0, the 1.1 version added six extra methods: \verb|PUT|, \verb|PATCH|, \verb|DELETE|, \verb|CONNECT|, \verb|TRACE|, \& \verb|OPTIONS|.
\end{itemize}

With persistent connections, multiple requests could share the same connection, but they still had to be serialised one after the other;
therefore, a client \& a server could only execute a single request/response exchange at any given time for each connection.
As the web evolved, more concurrency was required when fetching \& rendering web pages with a large number of resources (such as CSS, JavaScript, images, etc.).
The only way to gain concurrency at the network layer was to use multiple TCP connections to the same origin in parallel, but this has several negative effects:
\begin{itemize}
    \item   \textbf{Initial overhead:} establishing \& maintaining multiple connections requires additional overhead;
    \item   \textbf{Network congestion:} multiple parallel connections increase the amount of traffic on the network;
    \item   \textbf{Inefficient use of resources:} every TCP connection requires resources on both the client \& the server sides, such as buffers, sockets, \& memory; \&
    \item   \textbf{Fairness issues:} some users or applications may monopolise bandwidth by opening many TCP connections.
\end{itemize}

\subsubsection{SPDY}
An unofficial HTTP protocol named \textbf{SPDY} was developed by Google in 2009 as an experimental protocol to improve web performance.
SPDY opens one connection per domain;
multiple data streams are multiplexed over this single TCP connection for efficiency.
This reduces the amount of redundant header information sent every time a new page is requested.
Higher priority resources could be transferred faster than lower priority resources.
SPDY was supported by the Chrome browser and deployed in most Google services.

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/http11vsspdy.png}
    \caption{HTTP/1.1 versus SPDY}
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/spdyperformance.png}
    \caption{SPDY performance}
\end{figure}

\subsubsection{HTTP/2.0}
\textbf{HTTP/2.0} was officially released in 2015 (RFC 7540), about 18 years after HTTP/1.1.
HTTP/2.0 is largely derived from SPDY, and implements several new features to improve the protocol performance:
\begin{itemize}
    \item   \textbf{Request multiplexing:} HTTP/1.1 was a \textit{sequential protocol}, meaning that we can only send a single request at a time;
            HTTP/2.0 allows us to send requests \& receive responses asynchronously, thus allowing us to make multiple requests at the same time using a single connection.
            \begin{figure}[H]
                \centering
                \includegraphics[width=0.8\textwidth]{./images/http2action.png}
                \caption{HTTP/2.0 in action}
            \end{figure}

    \item   \textbf{Request prioritisation:} HTTP/2.0 allows us to set a priority for each request, thus allowing us to explicit about the order in which we expect responses, such as getting a webpage's CSS before its JavaScript file.

    \item   \textbf{Header compression:} headers are compressed with HPACK (RFC 7541), with 76\% compression of the ingress header, \& 69\% compression of the egress header.

    \item   \textbf{Server push:} to avoid a server receiving lots of requests, HTTP/2.0 introduced a server push functionality.
            With that, the server tries to predict the resources that will be requested soon.
            So, the server proactively pushes these resources to the client cache.
\end{itemize}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/httpversioncomparison.png}
    \caption{HTTP version comparison}
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/httpversioncomparison.png}
    \caption{HTTP version comparison}
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{./images/http1vshttp2.png}
    \caption{HTTP/1.0 versus HTTP/2.0 performance}
\end{figure}

Improvements at the application layer have been implemented in HTTP/2.0;
to further improve the performance, fundamental changes to the underlying transport layer are required.

\subsection{Transport Protocols}
The \textbf{Transmission Control Protocol (TCP)} ensures reliability \& flow control (by ensuring that data sent is delivered to the receiver application and ensuring that the receiver buffer doesn't overflow), ordered delivery (by ensuring bits pushed by the sender arrive at the receiver application in order), \& congestion control (by ensuring that the data sent doesn't overwhelm network resources).
\\\\
The \textbf{User Datagram Protocol (UDP)} facilitates abstraction of independent messages between endpoints.
It has minimal overhead, making it the recommended transport to meet the strict latency bounds of real-time applications, but provides limited support to applications.
There is no guarantee of delivery.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.8\textwidth]{./images/tcpvsudp.png}
    \caption{TCP versus UDP}
\end{figure}

\subsubsection{Issues with TCP}
\begin{itemize}
    \item   Slow connection establishment.
    \item   The TCP 3-way handshake has very high latency, particularly for small web requests.
            This problem is made worse by adding security (HTTP/TLS) over TCP: the TLS handshake adds more round trips.
    \item   Head-of-line blocking.
\end{itemize}

TCP is implemented in the kernel of the operating system, and to change TCP, you would likely have to change the operating system kernel on all servers \& clients around the world.
You might even need to change the entire network, as middleboxes may drop packets if they don't understand something on the packet.

\subsection{QUIC}
The \textbf{QUIC} protocol was developed to overcome the performance issues in TCP.
It is a reliable transport over UDP.
It was standardised in 2021, and in 2022 the IETF standardised HTTP/3 as RFC 9114 which uses QUIC by default.
As of 2025, around 27-30\% of all website support HTTP/3.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.8\textwidth]{./images/quicfit.png}
    \caption{Where does QUIC fit?}
\end{figure}

The main difference between HTTP/2.0 and HTTP/3.0 is the underlying transport layer protocol:
HTTP/2.0 uses TCP connections, while HTTP/3.0 uses QUIC (UDP-based) connections.
\\\\
QUIC features include:
\begin{itemize}
    \item   \textbf{Multiplexing without head-of-line blocking:} in QUIC, each byte stream is transport independently over the network.
            QUIC ensures the in-order delivery of packets within the same byte stream;
            if a packet gets lost, only the affected stream gets blocked and waits for its re-transmission.
    \item   \textbf{Low-latency connections:} WUIC has a faster connection setup, as it combines the transport handshake with the TLS cryptographic session establishment, which in TCP+TLS are two separate processes.
            To reduce the time required to establish a new connection, a client that has previously connected to a server may cache certain parameters from that connection and subsequently set up a \textbf{0-RTT connection} with the server;
            with the 0-RTT feature, a HTTP request can be sent, and a (partial) response can be received during the very first handshake.
    \item   \textbf{Connection migration:} QUIC improves performance during network-switching events, like what happens when a user of a mobile device moves from a local WiFi hotspot to a mobile network.
            When this occurs on TCP, a lengthy process starts where every existing connection times out one-by-one and is then re-established on demand.
            QUIC includes a connection identifier to uniquely identify the connection to the server regardless of source;
            this allows the connection to be re-established simply by sending a packet, which always contains this ID, as the original connection ID will still be valid even if the user's IP address changes.
    \item   \textbf{Linkability prevention: } to avoid privacy issues, e.g., to prevent hackers from following the physical movement of a smartphone user by tracking the unencrypted CID across networks, QUIC uses a list of connection identifiers instead of just one.
    \item   \textbf{Encryption:} TCP + TLS only encrypts the actual HTTP data.
            QUIC also encrypts large parts of the protocol header.
            Metadata, such as packet numbers and connection-close signals, which were visible to all middleboxes (\& attackers) in TCP, are now only available to the client \& server in QUIC.
    \item   \textbf{Resistance to protocol ossification:} protocol ossification is an inherent characteristic of protocols implemented in the oeprating system kernel, such as TCP;
            kernels are rarely updated, which applies even more to the operating systems of middleboxes, such as firewalss \& load balancers.
            It is a problem because it makes it hard to introduce new features, as middleboxes with an older version of the protoocl don't recognise the new feature and drop the packets.
            QUIC aims to solve this issue:
            it runs in the user space instead of the kernel, so it's easier to deploy new implementations.
            Furthermore, QUIC is heavily encrypted, so if a middlebox can't read a piece of information, it can't make any decisions based on that information.

    \item   \textbf{QPACK field compressoin:} QPACK is a field compression format for HTTP/3.0.
            Field compression eliminates redundant metadata by assigning indexes to fields that are used multiple times during the connection.
            The goal of QPACK is to reduce the space taken up by the HTTP headers:
            smaller size translates to higher throughput and better user experience.
            The effectiveness of data compression is expressed as a compression ratio.
\end{itemize}

A QUIC packet is composed of a common header followed by one or more frames.
\begin{figure}[H]
    \centering
    \includegraphics[width=0.8\textwidth]{./images/quicpacket.png}
    \caption{QUIC packet structure}
\end{figure}

The QUIC endpoints communicate by exchanging packets;
the QUIC packet consists of a header \& a payload.
QUIC has two different types of headers: the long header is used prior to the connection establishment, and the short header is used after the first connection is established.
Packets are carried in UDP datagrams.
\\\\
QUIC provides different packet types:
\begin{itemize}
    \item   \textbf{Initial packet:} transports the first CRYPTO frames transmitted by the client \& server during the key exchange and the ACK frames in both directions.
    \item   \textbf{Handshake packet:} used for sending \& receiving encrypted handshake messages \& acknowledgements between the server \& the client.
    \item   \textbf{0-RTT packet:} sends ``early'' data from the client to the server before the handshake is completed.
    \item   \textbf{1-RTT packet:} used to exchange data between the client \& the server after the handshake is completed.
\end{itemize}

\begin{figure}[H]
    \centering
    \includegraphics[width=0.8\textwidth]{./images/quickinitalpacket.png}
    \caption{QUIC initial packet example}
\end{figure}

A QUIC packet is composed of a common header followed by one or more frames.
There are various types of frames:
\begin{itemize}
    \item   \textbf{\texttt{ACK} frame:} receivers send \verb|ACK| frames to inform senders of packets they have received \& processed.
            The \verb|ACK| frame contains one or more \verb|ACK| ranges.

    \item   \textbf{\texttt{CRYPTO} frame:} used to transmit cryptographic handshake messages.

    \item   \textbf{\texttt{STREAM} frame:} implicitly creates a stream and carries stream data.
            It contains a Stream ID, Offset, Length, \& Stream Data.

    \item   \textbf{\texttt{MAX\_DATA} frame:} used in flow control to inform the peer of the maximum amount of data that can be sent on the connection as a whole.

    \item   \textbf{\texttt{MAX\_STREAM\_DATA} frame:} used in flow control to inform a peer of the maximum amount of data that can be sent on a stream.

    \item   \textbf{\texttt{MAX\_STREAMS} frame:} informs the peer of the cumulative number of streams of a give type it is permitted to open.
\end{itemize}

\begin{figure}[H]
    \centering
    \includegraphics[width=0.8\textwidth]{./images/quicframes.png}
    \caption{QUIC \texttt{CRYPTO} frame examples}
\end{figure}

\textbf{Streams} in QUIC provide a lightweight, ordered byte-stream abstraction to an application.
Streams can be created by either endpoint, and can concurrently send data interleaved with other streams.
Streams are identified within a connection by a numeric value, referred to as the stream ID (a 62-bit integer) that is unique for all streams on a connection.
Client-initiated streams have even-numbered stream IDs and server-initiated streams have odd-numbered stream IDs.

\end{document}

