diff --git a/year4/semester2/CT420/notes/CT420.pdf b/year4/semester2/CT420/notes/CT420.pdf index ca884aa6..fda9638b 100644 Binary files a/year4/semester2/CT420/notes/CT420.pdf and b/year4/semester2/CT420/notes/CT420.pdf differ diff --git a/year4/semester2/CT420/notes/CT420.tex b/year4/semester2/CT420/notes/CT420.tex index 10c6ee3b..b1bf8410 100644 --- a/year4/semester2/CT420/notes/CT420.tex +++ b/year4/semester2/CT420/notes/CT420.tex @@ -269,14 +269,320 @@ however, nothing can be said about the relationship between two events $a$ \& $b \subsubsection{Vector Clocks} In practice, causality is captured by means of \textbf{vector clocks}: -\begin{itemize} +\begin{enumerate} \item There is an ordered list of logical clocks, with one per process. - \item Each process $P_i$ maintains a vector $\vec{V_i}$, initially containing all zeroes. - \item On a local event $e$, $P_i$ increments $\vec{V_i}[i]$ (the $i$\textsuperscript{th} vector component). - If the event is ``message send'', a new $\vec{V_i}$ is copied into the packet. - \item If $P_i$ receives a message from $P_m$, then, for all $k$ + Each process $P_i$ maintains a vector $\vec{V_i}$, initially containing all zeroes. + Each index $k$ of a vector clock $\vec{V_i}[k]$ represents the number of events that process $P_i$ knows have occurred in process $P_k$. + $\vec{V_i}[i]$ is the count of events that have occurred locally at process $P_i$, while $\vec{V_i}[k]$ (for $k\neq i$) is the count of events in process $P_k$ that $P_i$ is aware of. + + \item On a local event $e$, $P_i$ increments its own clock component $\vec{V_i}[i]$. + If the event is ``message send'', a new $\vec{V_i}$ is copied into the packet, so that on message sends the current vector state is included in the message. + + \item If $P_i$ receives a message from $P_m$, then each index $\vec{V_i}[k]$ where $k \neq i$ is set to $\text{max}(\vec{V_m}[k], \vec{V_i}[k])$, and $\vec{V_i}[i]$ is incremented. +\end{enumerate} + +Intuitively, $\vec{V_i}[k]$is the number of events in process $P_k$ that have been observed by $P_i$. + +\begin{figure}[H] + \centering + \includegraphics[width=\textwidth]{./images/vector_clocks_example.png} + \caption{Vector clocks example} +\end{figure} + +In the above example, when process $P_2$ receives message $m_1$, it merges the entries from $P_1$'s clock by choosing the maximum value of each position. +Similarly, when $P_3$ receives $m_2$, it merges in $P_2$'s clock, thus incorporating the changes from $P_1$ that $P_2$ already saw. +Vector clocks explicitly track the transitive causal order: $f$'s timestamp captures the history of $a$, $b$, $c$, \& $d$. +\\\\ +To use vector clocks for ordering, we can compare them piecewise: +\begin{itemize} + \item We say $\vec{V_i} = \vec{V_j}$ if and only if $\vec{V_i}[k] = \vec{V_j}[k] \forall k$. + \item We say $\vec{V_i} \leq \vec{V_j}$ if and only if $\vec{V_i}[k] \leq \vec{V_j}[k] \forall k$. + \item We say $\vec{V_i} < \vec{V_j}$ if and only if $\vec{V_i} \leq \vec{V_j}$ and $\vec{V_i} \neq \vec{V_j}$. + \item We say $\vec{V_i} \sim \vec{V_j}$ otherwise, e.g., $\vec{V_i} = [2,0,0]$ and $\vec{V_j} = [0,0,1]$. \end{itemize} +For any two event timestamps $T(a)$ \& $T(b)$: +\begin{itemize} + \item if $a \rightarrow b$, then $T(a) < T(b)$; \textbf{and} + \item if $T(a) < T(b)$, then $a \rightarrow b$. +\end{itemize} + +Hence, we can use timestamps to determine if there is a causal ordering between any two events. + +\begin{figure}[H] + \centering + \includegraphics[width=\textwidth]{./images/vector_vs_lamport.png} + \caption{Lamport clocks versus vector clocks} +\end{figure} + +\section{The NTP Protocol} +The options for computer clocks are essentially as follows: +\begin{itemize} + \item \textbf{Option A:} stick to crystals. + \begin{itemize} + \item Costly precision manufacturing. + \item Works indoors. + \item Temperature Compensated Crystal Oscillator (TCXO). + \item Oven Controlled Crystal Oscillator (OCXO). + \end{itemize} + + \item \textbf{Option B:} buy an atomic clock (\$50,000 -- \$100,000) or a GNSS receiver (based on an atomic clock, but doesn't work indoors), or a time signal radio receiver if you are based in central Europe. + + \item \textbf{Option C:} use software-based approaches to discipline cheap crystal clocks. + Less quality, but useful for certain applications, and works indoors. +\end{itemize} + +\textbf{Distributed master clocks} provide a time reference to hosts that are inter-connected via a network. +The underlying time-synchronisation protocols combine aspects of Cristian's algorithm (i.e., RTD calculation) and Berkeley's algorithm (i.e., combining multiple reference time sources). +Good time synchronisation requires: +\begin{itemize} + \item Good time references: easily done with GPS, atomic clocks, etc. + \item Predictable / symmetric / deterministic network latencies: doable in LAN setups, but not guaranteed in Internet data communication. +\end{itemize} + +There are two main distributed master clock protocols: +\begin{itemize} + \item \textbf{Network Time Protocol (NTP):} defined in RFC 5905, originally used in the Unix-based NTP daemon, one of the first Internet protocols that ever evolved. + \item \textbf{Precision Time Protocol (PTP):} designed for managed networks, e.g., LAN. +\end{itemize} + +\begin{figure}[H] + \centering + \includegraphics[width=0.7\textwidth]{./images/ntpvsptp.png} + \caption{NTP \& PTP characteristics} +\end{figure} + +In \textbf{uni-directional synchronisation} a reference clock sends a timestamp to a host via a network. +The host then uses the timestamp to set its local clock. +Useful when message latencies are minor relative to the synchronisation levels required. +\\\\ +In \textbf{round-trip synchronisation (RTS)} a host sends a request message, receives a reference clock response message with known (local) submission \& arrival times, allowing for the calculation of the round-trip delay (1) and the host clock error (2), i.e., the phase offset. +Variations of RTS form the basis for NTP \& PTP. + +\begin{align} + \delta = (T_{i+3} - T_i) - (T_{i+2} - T_{i+1}) +\end{align} +\begin{align} + \theta = (T_{i+1}-T_i) + \frac{(T_{i+2}-T_{i+3})}{2} +\end{align} + +\begin{figure}[H] + \centering + \includegraphics[width=0.4\textwidth]{./images/rts.png} + \caption{ RTS example } +\end{figure} + +\subsection{NTP} +The NTP architecture, protocol, \& algorithms have evolved over the last 40 years to the latest NTP Version 4. +NTP is the standard Internet protocol for time synchronisation \& co-ordinated UTC time distribution. +It is a fault-tolerant protocol, as it automatically selects the best of several available time sources to synchronise with. +It is highly scalable, as the nodes form a hierarchical structure with reference clock(s) at the top: +\begin{itemize} + \item Stratum 0: Time Reference Source (e.g., GPS, TAI atomic clocks, DCF 77). + \item Stratum 1: Primary Time Server. +\end{itemize} + +NTP applies some general aforementioned principles such as avoiding setting clocks backwards and avoiding large step changes; +the required change (positive or negative) is amortised over a series of short intervals (e.g., over multiple ticks). +\\\\ +NTP is the longest-running and continuously operating Internet protocol (since around 1979). +Government agencies in many other countries and on all continents (including Antarctica) operate public NTP primary servers. +National \& regional service providers operate public NTP secondary servers synchronised to the primary servers. +Many government agencies, private \& public institutions including universities, broadcasters, financial institutions, \& corporations operate their own NTP networks. +\begin{figure}[H] + \centering + \includegraphics[width=0.5\textwidth]{./images/ntphierarchy.png} + \caption{ NTP hierarchy } +\end{figure} + +In \textbf{client/server mode}, UDP is used for data transfer (no TCP), i.e., NTP over UDP on UDP port 123. +There is also the optional use of broadcasting or multicasting (not covered here). +Several packet exchanges between the NTP client and the Stratum server take place to determine the client offset: +\begin{enumerate} + \item The client sends a packet with \textbf{originate timestamp} $A$. + \item The server receives the packet and returns a response containing the originate timestamp $A$ as well as the \textbf{receive timestamp} $B$ and the \textbf{transmit timestamp} $C$. + \item The client receives this packet and processes $A$, $B$, $C$, as well as the packet arrival time $D$ of the received packet; it then determines the offset and the round-trip delay (RTD). +\end{enumerate} + +\begin{figure}[H] + \centering + \includegraphics[width=0.7\textwidth]{./images/ntpoperation.png} + \caption{ NTP operation } +\end{figure} + +The above example is of a symmetric network with a 15ms delay each way; the client's clock lags 5ms behind the server's clock: +\begin{align*} + \text{RTD} = (D-A)-(C-B) = 32-2 =& 30\text{ms} \\ + \text{Offset} = \frac{((B-A)+(C-D))}{2} = \frac{(20 + (-10))}{2} =& 5\text{ms} +\end{align*} + +\begin{figure}[H] + \centering + \includegraphics[width=0.7\textwidth]{./images/networkdelayasymmetry.png} + \caption{ Network delay asymmetry } +\end{figure} + +In the above example, the client's clock still lags 5ms behind the servers clock, but there is an asymmetric network latency: 10ms versus 20ms: +\begin{align*} + \text{RTD} = (D-A)-(C-B) = 32-2 =& 30\text{ms} \\ + \text{Offset} = \frac{((B-A)+(C-D))}{2} = \frac{(15 + (-15))}{2} =& 0\text{ms} +\end{align*} + +Typical NTP performance for various set-ups is as follows: +\begin{itemize} + \item Small LAN: \sim10 microseconds best possible case on a 2-node LAN, \sim220 microseconds on a real-world small LAN. + \item Typical large-building LAN: \sim2ms. + \item Internet with a few hops: 10–20ms. + \item Long distance and/or slow or busy link: 100ms–1s. +\end{itemize} + +Accuracy is further degraded on networks with asymmetric traffic delays. +\\\\ +The \textbf{NTP time format} has a reference scale of UTC. +Time parameters are 64 bits long: +\begin{itemize} + \item Seconds since January 1, 1900 (32 bits, unsigned). + \item Fraction of second (32 bits, unsigned). +\end{itemize} +The NTP time format has a dynamic range of 136+ years, with rollover in 2036. +Its resolution is 2\textsuperscript{-32} seconds \sim232 picoseconds. + +\subsubsection{NTP Protocol Header} +\begin{figure}[H] + \centering + \includegraphics[width=0.7\textwidth]{./images/ntpheader.png} + \caption{ NTP protocol header } +\end{figure} +\begin{itemize} + \item \verb|LI| (\textbf{Leap Indicator}) 2-bit: + \begin{itemize} + \item \verb|0|: no warning; + \item \verb|1|: last minute of the day has 61 seconds; + \item \verb|2|: last minute of the day has 59 seconds. + \end{itemize} + + + \item \verb|VN| (\textbf{Version Number}) 2-bit: currently 4. + + \item \verb|Mode|: 3-bit integer, including: + \begin{itemize} + \item \verb|3|: client; + \item \verb|4|: server. + \end{itemize} + + \item \verb|Stratum|: 8-bit integer for Stratum server hierarchy level, including: + \begin{itemize} + \item \verb|1|: primary server (i.e., stratum 1); + \item \verb|2|–\verb|15|: secondary server. + \end{itemize} + + \item \verb|Poll|: 8-bit signed integer representing the maximum interval between successive message, in $\log_2$ seconds. + This field indicates the interval at which the client will poll the NTP server for time updates. + The client dynamically adjusts this interval based on its clock's stability \& the network conditions to balance accuracy \& network load. + + \item \verb|Precision|: 8-bit signed integer representing the resolution of the system clock (the tick increment) in $\log_2$ seconds; + e.g., a value of -18 corresponds to a resolution of about one microsecond (2\textsuperscript{-18}) seconds. + + \item \verb|Root Delay|: round-trip packet delay from a client to a stratum 1 server. + It gives a crude estimate of the worst-case time transfer error between a client and a stratum 1 server due to network asymmetry, i.e., if all of the round-trip delay was in one direction and none in the other direction. +\end{itemize} + +For a single client clock, the \textbf{dispersion} is a measure of how much the client's clock might drift during a synchronisation cycle: +\begin{align*} + \text{Dispersion} = \textit{DR} \times (D-A) + \textit{TS} +\end{align*} +where $(D-A)$ is the duration of a synchronisation cycle, with $A$ being the first timestamp and $D$ being the last timestamp, \textit{DR} being the local clock skew (i.e., the deviation of actual clock tick frequency to nominal clock tick frequency), and \textit{TS} being the timestamping errors due to the finite resolution of the clock and delays in reading the clock when fetching a timestamp. +\\\\ +The \textbf{root dispersion} of a client clock is the combined dispersions of all stratum servers along the path to a Stratum 1 server. +\\\\ +The \textbf{root distance} is the sum of root dispersion and half the root delay. +It provides a comprehensive measure of the maximum error in time synchronisation as the total worst case timing error accumulated between the Stratum 1 server and the client. + +\begin{figure}[H] + \centering + \includegraphics[width=0.7\textwidth]{./images/rootdispersion.png} + \caption{ Example root dispersion \& root delay } +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[width=0.7\textwidth]{./images/annotatedntpheader.png} + \caption{ Annotated NTP protocol header } +\end{figure} + +A \textbf{reference ID (refid)} is a 32-bit code (4 ASCII bytes) identifying the particular server or reference clock, e.g.,: +\begin{itemize} + \item \verb|GPS|: Global Positioning System; + \item \verb|GAL|: Galileo Positioning System; + \item \verb|PPS|: Generic Pulse-Per-Second; + \item \verb|DCF|: LF Radio DCF77 Mainflingen, DE 77.5 kHz; + \item \verb|WWV|: HF Radio WWV Fort Collins, Colorado; + \item \verb|GOOG|: unofficial Google refid used by Google NTP servers as \verb|time4.google.com|. +\end{itemize} + +\begin{figure}[H] + \centering + \includegraphics[width=0.7\textwidth]{./images/ntparchitecuraloverview.png} + \caption{ NTP architectural overview } +\end{figure} + +An NTP client synchronises with multiple stratum servers. +It uses a range of algorithms to deal with variable \& asymmetric non-deterministic network delays and to determine its most likely offset, thereby running a series of processes: +\begin{itemize} + \item The peer process runs when a packet is received; + \item The poll process sends packets at intervals determined by the clock discipline process \& remote server; + \item The system process runs when a new update is received; + \item The clock discipline process implements clock time adjustments; + \item The clock adjust process implements periodic clock frequency (VFO) adjustments. +\end{itemize} + +For each stratum there is a \textbf{poll process} that sends NTP packets at intervals ranging from 8 seconds to 36 hours. +The corresponding \textbf{peer processes} receive NTP packets and, after performing some packet sanity tests, $T1$ – $T4$ are determined / extracted. +The NTP daemon calculates offset \& delay as seen before. +The time series of offset \& delay values calculated by multiple peer processes are processed by a sequence of algorithms, thus eliminating servers with long RTD or servers that show ``strange'' offsets which, for example, are often the result of network asymmetries. + +\subsubsection{Mitigation Algorithms} +The \textbf{clock filter algorithm} uses a sliding window of eight samples for each stratum server and picks out the sample with the least expected error, i.e., it chooses the sample with the minimum RTD. +It is effective at removing spikes resulting from intermittent network congestions. + +\begin{figure}[H] + \centering + \includegraphics[width=0.7\textwidth]{./images/clockfilteralg.png} + \caption{ Clock filter algorithm before and after } +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[width=0.7\textwidth]{./images/clockfilteralgmotivation.png} + \caption{ + The wedge scattergram plots sample points of offset versus delay (RTD) collected over a 24-hour period by a client clock communicating with a single stratum server. + For this experiment, the client clock is externally synced to the stratum server, so the offset should be zero; + however, as the (network) round trip delay increases, the offset variability increases, resulting in increasingly larger offset errors. + Therefore, the best samples are those at the lowest delay. + This is taken into account by the clock filter algorithm. + } +\end{figure} + +The \textbf{intersection algorithm} selects a subset of peers (i.e., stratum servers) and identifies truechimers \& truetickers based on the intersection of confidence (offset) intervals (i.e., min/max offsets of a clock over $x$ readings determines its invterval). + +\begin{figure}[H] + \centering + \includegraphics[width=0.7\textwidth]{./images/intersectional.png} + \caption{ Plot range of offsets calculated by each peer with 1, 2, \& 3 overlapping} +\end{figure} + +\textbf{Marzullo's algorithm} is an agreement protocol for estimating accurate time from a number of noisy time sources by intersecting offset intervals; if some intervals don't intersect, it considers the intersection of the majority of intervals. +It eliminates false tickers. +\\\\ +The \textbf{clock cluster algorithm} processes the truechimers returned by the clock intersection algorithm. +It produces a list of survivors by eliminating truechimers that have a comparably large root delay \& root dispersion. +Finally, the \textbf{clock combining algorithm} averages the time offsets of survivors using their root dispersions as a weight, i.e., survivors with a small root dispersion have a higher weight. +\\\\ +The \textbf{combining algorithm} provides a final offset, so the client clock can be adjusted. +The UNIX Clock Model provides the kernel variable \verb|tickadj| which amortises the required change gradually by making adjustments every tick e.g., every 10 milliseconds. + + + diff --git a/year4/semester2/CT420/notes/images/annotatedntpheader.png b/year4/semester2/CT420/notes/images/annotatedntpheader.png new file mode 100644 index 00000000..7ec3229e Binary files /dev/null and b/year4/semester2/CT420/notes/images/annotatedntpheader.png differ diff --git a/year4/semester2/CT420/notes/images/clockfilteralg.png b/year4/semester2/CT420/notes/images/clockfilteralg.png new file mode 100644 index 00000000..049bb797 Binary files /dev/null and b/year4/semester2/CT420/notes/images/clockfilteralg.png differ diff --git a/year4/semester2/CT420/notes/images/clockfilteralgmotivation.png b/year4/semester2/CT420/notes/images/clockfilteralgmotivation.png new file mode 100644 index 00000000..6fabc08a Binary files /dev/null and b/year4/semester2/CT420/notes/images/clockfilteralgmotivation.png differ diff --git a/year4/semester2/CT420/notes/images/intersectional.png b/year4/semester2/CT420/notes/images/intersectional.png new file mode 100644 index 00000000..d8e4ff6a Binary files /dev/null and b/year4/semester2/CT420/notes/images/intersectional.png differ diff --git a/year4/semester2/CT420/notes/images/networkdelayasymmetry.png b/year4/semester2/CT420/notes/images/networkdelayasymmetry.png new file mode 100644 index 00000000..1ff5d05c Binary files /dev/null and b/year4/semester2/CT420/notes/images/networkdelayasymmetry.png differ diff --git a/year4/semester2/CT420/notes/images/ntparchitecuraloverview.png b/year4/semester2/CT420/notes/images/ntparchitecuraloverview.png new file mode 100644 index 00000000..8ddf35ca Binary files /dev/null and b/year4/semester2/CT420/notes/images/ntparchitecuraloverview.png differ diff --git a/year4/semester2/CT420/notes/images/ntpheader.png b/year4/semester2/CT420/notes/images/ntpheader.png new file mode 100644 index 00000000..c247272c Binary files /dev/null and b/year4/semester2/CT420/notes/images/ntpheader.png differ diff --git a/year4/semester2/CT420/notes/images/ntphierarchy.png b/year4/semester2/CT420/notes/images/ntphierarchy.png new file mode 100644 index 00000000..0138152d Binary files /dev/null and b/year4/semester2/CT420/notes/images/ntphierarchy.png differ diff --git a/year4/semester2/CT420/notes/images/ntpoperation.png b/year4/semester2/CT420/notes/images/ntpoperation.png new file mode 100644 index 00000000..7ffd2bd8 Binary files /dev/null and b/year4/semester2/CT420/notes/images/ntpoperation.png differ diff --git a/year4/semester2/CT420/notes/images/ntpvsptp.png b/year4/semester2/CT420/notes/images/ntpvsptp.png new file mode 100644 index 00000000..1222011b Binary files /dev/null and b/year4/semester2/CT420/notes/images/ntpvsptp.png differ diff --git a/year4/semester2/CT420/notes/images/rootdispersion.png b/year4/semester2/CT420/notes/images/rootdispersion.png new file mode 100644 index 00000000..ecc2c792 Binary files /dev/null and b/year4/semester2/CT420/notes/images/rootdispersion.png differ diff --git a/year4/semester2/CT420/notes/images/rts.png b/year4/semester2/CT420/notes/images/rts.png new file mode 100644 index 00000000..fa0d8dd5 Binary files /dev/null and b/year4/semester2/CT420/notes/images/rts.png differ diff --git a/year4/semester2/CT420/notes/images/vector_clocks_example.png b/year4/semester2/CT420/notes/images/vector_clocks_example.png new file mode 100644 index 00000000..dac1b9a0 Binary files /dev/null and b/year4/semester2/CT420/notes/images/vector_clocks_example.png differ diff --git a/year4/semester2/CT420/notes/images/vector_vs_lamport.png b/year4/semester2/CT420/notes/images/vector_vs_lamport.png new file mode 100644 index 00000000..621b05ed Binary files /dev/null and b/year4/semester2/CT420/notes/images/vector_vs_lamport.png differ