[CT420]: Add WK02-2 lecture notes

year4/semester2/CT420/notes/CT420.tex

@@ -269,14 +269,320 @@ however, nothing can be said about the relationship between two events $a$ \& $b
 
 \subsubsection{Vector Clocks}
 In practice, causality is captured by means of \textbf{vector clocks}:
-\begin{itemize}
+\begin{enumerate}
     \item   There is an ordered list of logical clocks, with one per process.
-    \item   Each process $P_i$ maintains a vector $\vec{V_i}$, initially containing all zeroes.
+            Each process $P_i$ maintains a vector $\vec{V_i}$, initially containing all zeroes.
-    \item   On a local event $e$, $P_i$ increments $\vec{V_i}[i]$ (the $i$\textsuperscript{th} vector component).
+            Each index $k$ of a vector clock $\vec{V_i}[k]$ represents the number of events that process $P_i$ knows have occurred in process $P_k$.
-            If the event is ``message send'', a new $\vec{V_i}$ is copied into the packet.
+            $\vec{V_i}[i]$  is the count of events that have occurred locally at process $P_i$, while $\vec{V_i}[k]$ (for $k\neq i$) is the count of events in process $P_k$ that $P_i$ is aware of.
-    \item   If $P_i$ receives a message from $P_m$, then, for all $k$
+
+    \item   On a local event $e$, $P_i$ increments its own clock component $\vec{V_i}[i]$.
+            If the event is ``message send'', a new $\vec{V_i}$ is copied into the packet, so that on message sends the current vector state is included in the message.
+
+    \item   If $P_i$ receives a message from $P_m$, then each index $\vec{V_i}[k]$ where $k \neq i$ is set to $\text{max}(\vec{V_m}[k], \vec{V_i}[k])$, and $\vec{V_i}[i]$ is incremented.
+\end{enumerate}
+
+Intuitively, $\vec{V_i}[k]$is the number of events in process $P_k$ that have been observed by $P_i$.
+
+\begin{figure}[H]
+    \centering
+    \includegraphics[width=\textwidth]{./images/vector_clocks_example.png}
+    \caption{Vector clocks example}
+\end{figure}
+
+In the above example, when process $P_2$ receives message $m_1$, it merges the entries from $P_1$'s clock by choosing the maximum value of each position.
+Similarly, when $P_3$ receives $m_2$, it merges in $P_2$'s clock, thus incorporating the changes from $P_1$ that $P_2$ already saw. 
+Vector clocks explicitly track the transitive causal order: $f$'s timestamp captures the history of $a$, $b$, $c$, \& $d$.
+\\\\
+To use vector clocks for ordering, we can compare them piecewise:
+\begin{itemize}
+    \item   We say $\vec{V_i} = \vec{V_j}$ if and only if $\vec{V_i}[k] = \vec{V_j}[k] \forall k$.
+    \item   We say $\vec{V_i} \leq \vec{V_j}$ if and only if $\vec{V_i}[k] \leq \vec{V_j}[k] \forall k$.
+    \item   We say $\vec{V_i} < \vec{V_j}$ if and only if $\vec{V_i} \leq \vec{V_j}$ and $\vec{V_i} \neq \vec{V_j}$.
+    \item   We say $\vec{V_i} \sim \vec{V_j}$ otherwise, e.g., $\vec{V_i} = [2,0,0]$ and $\vec{V_j} = [0,0,1]$.
 \end{itemize}
 
+For any two event timestamps $T(a)$ \& $T(b)$:
+\begin{itemize}
+    \item   if $a \rightarrow b$, then $T(a) < T(b)$; \textbf{and}
+    \item   if $T(a) < T(b)$, then $a \rightarrow b$.
+\end{itemize}
+
+Hence, we can use timestamps to determine if there is a causal ordering between any two events.
+
+\begin{figure}[H]
+    \centering
+    \includegraphics[width=\textwidth]{./images/vector_vs_lamport.png}
+    \caption{Lamport clocks versus vector clocks}
+\end{figure}
+
+\section{The NTP Protocol}
+The options for computer clocks are essentially as follows:
+\begin{itemize}
+    \item   \textbf{Option A:} stick to crystals.
+            \begin{itemize}
+                \item   Costly precision manufacturing.
+                \item   Works indoors.
+                \item   Temperature Compensated Crystal Oscillator (TCXO).
+                \item   Oven Controlled Crystal Oscillator (OCXO).
+            \end{itemize}
+
+    \item   \textbf{Option B:} buy an atomic clock (\$50,000 -- \$100,000) or a GNSS receiver (based on an atomic clock, but doesn't work indoors), or a time signal radio receiver if you are based in central Europe.
+
+    \item   \textbf{Option C:} use software-based approaches to discipline cheap crystal clocks.
+            Less quality, but useful for certain applications, and works indoors.
+\end{itemize}
+
+\textbf{Distributed master clocks} provide a time reference to hosts that are inter-connected via a network.
+The underlying time-synchronisation protocols combine aspects of Cristian's algorithm (i.e., RTD calculation) and Berkeley's algorithm (i.e., combining multiple reference time sources).
+Good time synchronisation requires:
+\begin{itemize}
+    \item   Good time references: easily done with GPS, atomic clocks, etc.
+    \item   Predictable / symmetric / deterministic network latencies: doable in LAN setups, but not guaranteed in Internet data communication.
+\end{itemize}
+
+There are two main distributed master clock protocols:
+\begin{itemize}
+    \item   \textbf{Network Time Protocol (NTP):} defined in RFC 5905, originally used in the Unix-based NTP daemon, one of the first Internet protocols that ever evolved.
+    \item   \textbf{Precision Time Protocol (PTP):} designed for managed networks, e.g., LAN.
+\end{itemize}
+
+\begin{figure}[H]
+    \centering
+    \includegraphics[width=0.7\textwidth]{./images/ntpvsptp.png}
+    \caption{NTP \& PTP characteristics}
+\end{figure}
+
+In \textbf{uni-directional synchronisation} a reference clock sends a timestamp to a host via a network.
+The host then uses the timestamp to set its local clock.
+Useful when message latencies are minor relative to the synchronisation levels required.
+\\\\
+In \textbf{round-trip synchronisation (RTS)} a host sends a request message, receives a reference clock response message with known (local) submission \& arrival times, allowing for the calculation of the round-trip delay (1) and the host clock error (2), i.e., the phase offset.
+Variations of RTS form the basis for NTP \& PTP.
+
+\begin{align}
+    \delta = (T_{i+3} - T_i) - (T_{i+2} - T_{i+1})
+\end{align}
+\begin{align}
+    \theta = (T_{i+1}-T_i) + \frac{(T_{i+2}-T_{i+3})}{2}
+\end{align}
+
+\begin{figure}[H]
+    \centering
+    \includegraphics[width=0.4\textwidth]{./images/rts.png}
+    \caption{ RTS example }
+\end{figure}
+
+\subsection{NTP}
+The NTP architecture, protocol, \& algorithms have evolved over the last 40 years to the latest NTP Version 4.
+NTP is the standard Internet protocol for time synchronisation \& co-ordinated UTC time distribution.
+It is a fault-tolerant protocol, as it automatically selects the best of several available time sources to synchronise with.
+It is highly scalable, as the nodes form a hierarchical structure with reference clock(s) at the top:
+\begin{itemize}
+    \item   Stratum 0: Time Reference Source (e.g., GPS, TAI atomic clocks, DCF 77).
+    \item   Stratum 1: Primary Time Server.
+\end{itemize}
+
+NTP applies some general aforementioned principles such as avoiding setting clocks backwards and avoiding large step changes;
+the required change (positive or negative) is amortised over a series of short intervals (e.g., over multiple ticks).
+\\\\
+NTP is the longest-running and continuously operating Internet protocol (since around 1979).
+Government agencies in many other countries and on all continents (including Antarctica) operate public NTP primary servers.
+National \& regional service providers operate public NTP secondary servers synchronised to the primary servers.
+Many government agencies, private \& public institutions including universities, broadcasters, financial institutions, \& corporations operate their own NTP networks. 
+\begin{figure}[H]
+    \centering
+    \includegraphics[width=0.5\textwidth]{./images/ntphierarchy.png}
+    \caption{ NTP hierarchy }
+\end{figure}
+
+In \textbf{client/server mode}, UDP is used for data transfer (no TCP), i.e., NTP over UDP on UDP port 123.
+There is also the optional use of broadcasting or multicasting (not covered here).
+Several packet exchanges between the NTP client and the Stratum server take place to determine the client offset:
+\begin{enumerate}
+    \item   The client sends a packet with \textbf{originate timestamp} $A$.
+    \item   The server receives the packet and returns a response containing the originate timestamp $A$ as well as the \textbf{receive timestamp} $B$ and the \textbf{transmit timestamp} $C$.
+    \item   The client receives this packet and processes $A$, $B$, $C$, as well as the packet arrival time $D$ of the received packet; it then determines the offset and the round-trip delay (RTD).
+\end{enumerate}
+
+\begin{figure}[H]
+    \centering
+    \includegraphics[width=0.7\textwidth]{./images/ntpoperation.png}
+    \caption{ NTP operation }
+\end{figure}
+
+The above example is of a symmetric network with a 15ms delay each way; the client's clock lags 5ms behind the server's clock:
+\begin{align*}
+    \text{RTD}      = (D-A)-(C-B) = 32-2 =& 30\text{ms} \\
+    \text{Offset}   = \frac{((B-A)+(C-D))}{2} = \frac{(20 + (-10))}{2} =& 5\text{ms}
+\end{align*}
+
+\begin{figure}[H]
+    \centering
+    \includegraphics[width=0.7\textwidth]{./images/networkdelayasymmetry.png}
+    \caption{ Network delay asymmetry }
+\end{figure}
+
+In the above example, the client's clock still lags 5ms behind the servers clock, but there is an asymmetric network latency: 10ms versus 20ms:
+\begin{align*}
+    \text{RTD}      = (D-A)-(C-B) = 32-2                                =& 30\text{ms} \\
+    \text{Offset}   = \frac{((B-A)+(C-D))}{2} = \frac{(15 + (-15))}{2}  =& 0\text{ms}
+\end{align*}
+
+Typical NTP performance for various set-ups is as follows:
+\begin{itemize}
+    \item   Small LAN: \sim10 microseconds best possible case on a 2-node LAN, \sim220 microseconds on a real-world small LAN.
+    \item   Typical large-building LAN: \sim2ms.
+    \item   Internet with a few hops: 10–20ms.
+    \item   Long distance and/or slow or busy link: 100ms–1s.
+\end{itemize}
+
+Accuracy is further degraded on networks with asymmetric traffic delays.
+\\\\
+The \textbf{NTP time format} has a reference scale of UTC.
+Time parameters are 64 bits long:
+\begin{itemize}
+    \item   Seconds since January 1, 1900 (32 bits, unsigned).
+    \item   Fraction of second (32 bits, unsigned).
+\end{itemize}
+The NTP time format has a dynamic range of 136+ years, with rollover in 2036.
+Its resolution is 2\textsuperscript{-32} seconds \sim232 picoseconds.
+
+\subsubsection{NTP Protocol Header}
+\begin{figure}[H]
+    \centering
+    \includegraphics[width=0.7\textwidth]{./images/ntpheader.png}
+    \caption{ NTP protocol header }
+\end{figure}
+\begin{itemize}
+    \item   \verb|LI| (\textbf{Leap Indicator}) 2-bit:
+            \begin{itemize}
+                \item   \verb|0|: no warning;
+                \item   \verb|1|: last minute of the day has 61 seconds;
+                \item   \verb|2|: last minute of the day has 59 seconds.
+            \end{itemize}
+
+
+    \item   \verb|VN| (\textbf{Version Number}) 2-bit: currently 4. 
+
+    \item   \verb|Mode|: 3-bit integer, including:
+            \begin{itemize}
+                \item   \verb|3|: client;
+                \item   \verb|4|: server.
+            \end{itemize}
+
+    \item   \verb|Stratum|: 8-bit integer for Stratum server hierarchy level, including:
+            \begin{itemize}
+                \item   \verb|1|: primary server (i.e., stratum 1);
+                \item   \verb|2|–\verb|15|: secondary server.
+            \end{itemize}
+
+    \item   \verb|Poll|: 8-bit signed integer representing the maximum interval between successive message, in $\log_2$ seconds.
+            This field indicates the interval at which the client will poll the NTP server for time updates.
+            The client dynamically adjusts this interval based on its clock's stability \& the network conditions to balance accuracy \& network load.
+
+    \item   \verb|Precision|: 8-bit signed integer representing the resolution of the system clock (the tick increment) in $\log_2$ seconds;
+            e.g., a value of -18 corresponds to a resolution of about one microsecond (2\textsuperscript{-18}) seconds.
+
+    \item   \verb|Root Delay|: round-trip packet delay from a client to a stratum 1 server.
+            It gives a crude estimate of the worst-case time transfer error between a client and a stratum 1 server due to network asymmetry, i.e., if all of the round-trip delay was in one direction and none in the other direction.
+\end{itemize}
+
+For a single client clock, the \textbf{dispersion} is a measure of how much the client's clock might drift during a synchronisation cycle:
+\begin{align*}
+    \text{Dispersion} = \textit{DR} \times (D-A) + \textit{TS}
+\end{align*}
+where $(D-A)$ is the duration of a synchronisation cycle, with $A$ being the first timestamp and $D$ being the last timestamp, \textit{DR} being the local clock skew (i.e., the deviation of actual clock tick frequency to nominal clock tick frequency), and \textit{TS} being the timestamping errors due to the finite resolution of the clock and delays in reading the clock when fetching a timestamp.
+\\\\
+The \textbf{root dispersion} of a client clock is the combined dispersions of all stratum servers along the path to a Stratum 1 server.
+\\\\
+The \textbf{root distance} is the sum of root dispersion and half the root delay.
+It provides a comprehensive measure of the maximum error in time synchronisation as the total worst case timing error accumulated between the Stratum 1 server and the client.
+
+\begin{figure}[H]
+    \centering
+    \includegraphics[width=0.7\textwidth]{./images/rootdispersion.png}
+    \caption{ Example root dispersion \& root delay }
+\end{figure}
+
+\begin{figure}[H]
+    \centering
+    \includegraphics[width=0.7\textwidth]{./images/annotatedntpheader.png}
+    \caption{ Annotated NTP protocol header }
+\end{figure}
+
+A \textbf{reference ID (refid)} is a 32-bit code (4 ASCII bytes) identifying the particular server or reference clock, e.g.,:
+\begin{itemize}
+    \item   \verb|GPS|: Global Positioning System;
+    \item   \verb|GAL|: Galileo Positioning System;
+    \item   \verb|PPS|: Generic Pulse-Per-Second;
+    \item   \verb|DCF|: LF Radio DCF77 Mainflingen, DE 77.5 kHz;
+    \item   \verb|WWV|: HF Radio WWV Fort Collins, Colorado;
+    \item   \verb|GOOG|: unofficial Google refid used by Google NTP servers as \verb|time4.google.com|.
+\end{itemize}
+
+\begin{figure}[H]
+    \centering
+    \includegraphics[width=0.7\textwidth]{./images/ntparchitecuraloverview.png}
+    \caption{ NTP architectural overview }
+\end{figure}
+
+An NTP client synchronises with multiple stratum servers.
+It uses a range of algorithms to deal with variable \& asymmetric non-deterministic network delays and to determine its most likely offset, thereby running a series of processes:
+\begin{itemize}
+    \item   The peer process runs when a packet is received;
+    \item   The poll process sends packets at intervals determined by the clock discipline process \& remote server;
+    \item   The system process runs when a new update is received;
+    \item   The clock discipline process implements clock time adjustments;
+    \item   The clock adjust process implements periodic clock frequency (VFO) adjustments.
+\end{itemize}
+
+For each stratum there is a \textbf{poll process} that sends NTP packets at intervals ranging from 8 seconds to 36 hours.
+The corresponding \textbf{peer processes} receive NTP packets and, after performing some packet sanity tests, $T1$ – $T4$ are determined / extracted.
+The NTP daemon calculates offset \& delay as seen before.
+The time series of offset \& delay values calculated by multiple peer processes are processed by a sequence of algorithms, thus eliminating servers with long RTD or servers that show ``strange'' offsets which, for example, are often the result of network asymmetries.
+
+\subsubsection{Mitigation Algorithms}
+The \textbf{clock filter algorithm} uses a sliding window of eight samples for each stratum server and picks out the sample with the least expected error, i.e., it chooses the sample with the minimum RTD.
+It is effective at removing spikes resulting from intermittent network congestions.
+
+\begin{figure}[H]
+    \centering
+    \includegraphics[width=0.7\textwidth]{./images/clockfilteralg.png}
+    \caption{ Clock filter algorithm before and after }
+\end{figure}
+
+\begin{figure}[H]
+    \centering
+    \includegraphics[width=0.7\textwidth]{./images/clockfilteralgmotivation.png}
+    \caption{ 
+        The wedge scattergram plots sample points of offset versus delay (RTD) collected over a 24-hour period by a client clock communicating with a single stratum server.
+        For this experiment, the client clock is externally synced to the stratum server, so the offset should be zero;
+        however, as the (network) round trip delay increases, the offset variability increases, resulting in increasingly larger offset errors.
+        Therefore, the best samples are those at the lowest delay.
+        This is taken into account by the clock filter algorithm.
+    }
+\end{figure}
+
+The \textbf{intersection algorithm} selects a subset of peers (i.e., stratum servers) and identifies truechimers \& truetickers based on the intersection of confidence (offset) intervals (i.e., min/max offsets of a clock over $x$ readings determines its invterval).
+
+\begin{figure}[H]
+    \centering
+    \includegraphics[width=0.7\textwidth]{./images/intersectional.png}
+    \caption{ Plot range of offsets calculated by each peer with 1, 2, \& 3 overlapping}
+\end{figure}
+
+\textbf{Marzullo's algorithm} is an agreement protocol for estimating accurate time from a number of noisy time sources by intersecting offset intervals; if some intervals don't intersect, it considers the intersection of the majority of intervals.
+It eliminates false tickers.
+\\\\
+The \textbf{clock cluster algorithm} processes the truechimers returned by the clock intersection algorithm.
+It produces a list of survivors by eliminating truechimers that have a comparably large root delay \& root dispersion.
+Finally, the \textbf{clock combining algorithm} averages the time offsets of survivors using their root dispersions as a weight, i.e., survivors with a small root dispersion have a higher weight.
+\\\\
+The \textbf{combining algorithm} provides a final offset, so the client clock can be adjusted.
+The UNIX Clock Model provides the kernel variable \verb|tickadj| which amortises the required change gradually by making adjustments every tick e.g., every 10 milliseconds.
+
+
+