% $Id: template.tex 11 2007-04-03 22:25:53Z jpeltier $

\documentclass{vgtc}                          % final (conference style)
%\documentclass[review]{vgtc}                 % review
%\documentclass[widereview]{vgtc}             % wide-spaced review
%\documentclass[preprint]{vgtc}               % preprint
%\documentclass[electronic]{vgtc}             % electronic version

%% Uncomment one of the lines above depending on where your paper is
%% in the conference process. ``review'' and ``widereview'' are for review
%% submission, ``preprint'' is for pre-publication, and the final version
%% doesn't use a specific qualifier. Further, ``electronic'' includes
%% hyperreferences for more convenient online viewing.

%% Please use one of the ``review'' options in combination with the
%% assigned online id (see below) ONLY if your paper uses a double blind
%% review process. Some conferences, like IEEE Vis and InfoVis, have NOT
%% in the past.

%% Figures should be in CMYK or Grey scale format, otherwise, colour 
%% shifting may occur during the printing process.

%% These few lines make a distinction between latex and pdflatex calls and they
%% bring in essential packages for graphics and font handling.
%% Note that due to the \DeclareGraphicsExtensions{} call it is no longer necessary
%% to provide the the path and extension of a graphics file:
%% \includegraphics{diamondrule} is completely sufficient.
%%
\ifpdf%                                % if we use pdflatex
  \pdfoutput=1\relax                   % create PDFs from pdfLaTeX
  \pdfcompresslevel=9                  % PDF Compression
  \pdfoptionpdfminorversion=7          % create PDF 1.7
  \ExecuteOptions{pdftex}
  \usepackage{graphicx}                % allow us to embed graphics files
  \DeclareGraphicsExtensions{.pdf,.png,.jpg,.jpeg} % for pdflatex we expect .pdf, .png, or .jpg files
\else%                                 % else we use pure latex
  \ExecuteOptions{dvips}
  \usepackage{graphicx}                % allow us to embed graphics files
  \DeclareGraphicsExtensions{.eps}     % for pure latex we expect eps files
\fi%

%% it is recomended to use ``\autoref{sec:bla}'' instead of ``Fig.~\ref{sec:bla}''
\graphicspath{{figures/}{pictures/}{images/}{./}} % where to search for the images

\usepackage{microtype}                 % use micro-typography (slightly more compact, better to read)
\PassOptionsToPackage{warn}{textcomp}  % to address font issues with \textrightarrow
\usepackage{textcomp} % use better special symbols
%\usepackage{subfig}
\usepackage{subfigure}
\usepackage{subcaption}
\usepackage{mathptmx} 
\usepackage{amsmath}
% use matching math font
\usepackage{times}                     % we use Times as the main font
\renewcommand*\ttdefault{txtt}         % a nicer typewriter font
\usepackage{cite}                      % needed to automatically sort the references
\usepackage{tabu}                      % only used for the table example
\usepackage{booktabs}                  % only used for the table example
%% We encourage the use of mathptmx for consistent usage of times font
%% throughout the proceedings. However, if you encounter conflicts
%% with other math-related packages, you may want to disable it.


%% If you are submitting a paper to a conference for review with a double
%% blind reviewing process, please replace the value ``0'' below with your
%% OnlineID. Otherwise, you may safely leave it at ``0''.
\onlineid{0}

%% declare the category of your paper, only shown in review mode
\vgtccategory{Research}

%% allow for this line if you want the electronic option to work properly
\vgtcinsertpkg

%% In preprint mode you may define your own headline. If not, the default IEEE copyright message will appear in preprint mode.
%\preprinttext{To appear in an IEEE VGTC sponsored conference.}

%% This adds a link to the version of the paper on IEEEXplore
%% Uncomment this line when you produce a preprint version of the article 
%% after the article receives a DOI for the paper from IEEE
%\ieeedoi{xx.xxxx/TVCG.201x.xxxxxxx}


%% Paper title.

\title{High-Dimensional Data Exploration via Autoencoder Latent Spaces}

%% This is how authors are specified in the conference style

%% Author and Affiliation (single author).
%%\author{Roy G. Biv\thanks{e-mail: roy.g.biv@aol.com}}
%%\affiliation{\scriptsize Allied Widgets Research}

%% Author and Affiliation (multiple authors with single affiliations).
%%\author{Roy G. Biv\thanks{e-mail: roy.g.biv@aol.com} %
%%\and Ed Grimley\thanks{e-mail:ed.grimley@aol.com} %
%%\and Martha Stewart\thanks{e-mail:martha.stewart@marthastewart.com}}
%%\affiliation{\scriptsize Martha Stewart Enterprises \\ Microsoft Research}

%% Author and Affiliation (multiple authors with multiple affiliations)
%% Author and Affiliation (multiple authors with multiple affiliations)
\author{Maria Walch\thanks{e-mail: walch@rhrk.uni-kl.de}\\ %
     \parbox{1.4in}{\scriptsize \centering TU Kaiserslautern \\ IAV GmbH}
      \and Amala Paulson\thanks{e-mail: amala.paulson@iav.de}\\ %
      \scriptsize TU Kaiserslautern %
      \and Dirk J. Lehmann\thanks{e-mail: dirk.lehmann@iav.de}\\ %
        \scriptsize University of Magdeburg \\ Ostfalia University of Applied Sciences \\ IAV GmbH
      }
%\and Dirk J. Lehmann\thanks{e-mail: dirk.lehmann@iav.de}\\ %
%        \scriptsize University of Magdeburg \\ Ostfalia University of Applied Sciences \\ IAV GmbH %
%        }}

%% A teaser figure can be included as follows
\teaser{
  \centering
  \begin{minipage}[b]{0.36\textwidth}
  \includegraphics[width=\linewidth]{figures/lorenz.png}
      \caption{Lorenz Attractor Phase Diagram}
      \label{fig:teaser1}
  \end{minipage}
    \hfill
  \begin{minipage}[b]{0.30\textwidth}
    \includegraphics[width=\textwidth]{figures/labelled_UMAP_default_Lorenz.png}
    \caption{UMAP Embedding}
    \label{fig:teaser2}
  \end{minipage}
      \hfill
  \begin{minipage}[b]{0.30\textwidth}
    \includegraphics[width=\textwidth]{figures/labelled_t-SNE_default_Lorenz.png}
    \caption{TSNE Embedding}
    \label{fig:teaser3}
  \end{minipage}

  \label{fig:teaser}
}

%% Abstract section.
\abstract{Autoencoders are generative models that produce a low dimensional representation of the input data within their latent spaces. This representation is expected to contain all relevant informative features of the source data. In mathematical terms, informativeness of a representation means that the structure of the data is preserved in a - potentially functorial - manner. We consider three dimensional embeddings of the latent representation, performed with UMAP and TSNE to investigate this functoriality. For the visual analysis, we use clustering methods: DBSCAN, OPTICS and k-MEANS. We call the respective parameters \textit{metaparameters} and tune these in a cross-correlative way, in order to gain as much information about the latent spaces from the visual low dimensional embedding as possible. We apply our ansatz to the Lorenz attractor, the ECG5000 and also the DeepValve dataset.
} 
% end of abstract

%% ACM Computing Classification System (CCS). 
%% See <http://www.acm.org/about/class> for details.
%% We recommend the 2012 system <http://www.acm.org/about/class/class/2012>
%% For the 2012 system use the ``\CCScatTwelve'' which command takes four arguments.
%% The 1998 system <http://www.acm.org/about/class/class/2012> is still possible
%% For the 1998 system use the ``\CCScat'' which command takes four arguments.
%% In both cases the last two arguments (1998) or last three (2012) can be empty.

\CCScatlist{
  \CCScatTwelve{Functoriality}{Visu\-al\-iza\-tion}{Visu\-al\-iza\-tion techniques}{Embedding};
  \CCScatTwelve{Representation}{Dynamical System Analysis}{Visualization design and evaluation methods}{}
}

%\CCScatlist{
  %\CCScat{H.5.2}{User Interfaces}{User Interfaces}{Graphical user interfaces (GUI)}{};
  %\CCScat{H.5.m}{Information Interfaces and Presentation}{Miscellaneous}{}{}
%}

%% Copyright space is enabled by default as required by guidelines.
%% It is disabled by the 'review' option or via the following command:
% \nocopyrightspace

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%% START OF THE PAPER %%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

%% The ``\maketitle'' command must be the first command after the
%% ``\begin{document}'' command. It prepares and prints the title block.

%% the only exception to this rule is the \firstsection command
\firstsection{Introduction}

\maketitle

The intuition and expectation is brought to Autoencoders (AEs) \cite{bank2021autoencoders}, that their latent spaces should optimally consist a physical model \cite{katok} and represent those parts of the data, which are constitutive for its shaping dynamics. Underlying our visual analysis is the idea and perspective, that the visual investigation of dynamics in the latent space should at best lead to a dictionary-like understanding of fundamental dynamical properties such as chaos, fractality, autocorrelations, instability, seasonality, stationarity, global and local trends, cyclicity and other unexpected variations. An immediate use-case would be unsupervised time series segmentation, but the wide range of AE application to unsupervised data organization and interpretation itself also creates the need for comprehensability and reproducability of the results within the latent space (LHC: \cite{canelli2021autoencoders}, \cite{chekanov2021eventbased}; neutron scattering: \cite{samarakoon2021integration}; breast cancer subtype classification: \cite{rajpal2021deep}; exoplanet transitions: \cite{hoenes2021automatically}; anomaly detection in smart farming: \cite{adkisson2021autoencoderbased}). All applications are based on the hypothesis that Autoencoders can learn the true structure of the data. Two factors are crucial to whether this is actually the case for a concrete dataset:
\begin{enumerate}
    \item The chosen AE architecture; and
    \item its generalization capacity \cite{neyshabur2017exploring}.
\end{enumerate}

We investigated this hypothesis on the Lorenz dataset \cite{Lorenz} as a test example, the ECG5000 dataset \cite{ecg5000} and the Deepvalve (\cite{Ohe18}) dataset. We paid attention to the first of the above mentioned two factors by also testing different autoencoder architectures, namely vanilla AEs, convolutional AEs (CAES) \cite{Zhang2018ABA} and topological AEs \cite{Moortopae}.

The teaser images fig.~\ref{fig:teaser1}-\ref{fig:teaser3} depict two dimensional projections of the standard butterfly Lorenz attractor ($\sigma = 10, \rho = 28$ and $\beta = 8/3$). Fig.~\ref{fig:teaser2} and fig.~\ref{fig:teaser3} show the two dimensional UMAP \cite{UMAP} and TSNE \cite{tSNE} embedding of an 8 dimensional vanilla Autoencoder latent embedding. Although, with some imagination, the double wing structure of the 3D input representation can be recognized in fig.~\ref{fig:teaser2}, the TSNE embedding in fig.~\ref{fig:teaser3} is not interpretable a priori. The aim of our study is to develop a visual understanding about dynamical data and its latent representation and the interaction of data and Autoencoder architectures. We investigate this using a selection of the most popular clustering and embedding methods and develop a cross-correlative pipeline to get the most possible information from them.

\section{Related Work}
In \cite{Spinner2018Towar-43657} the authors visualize and compare the latent representations of the MNIST dataset \cite{inbookMNIST} processed within a Variational AE \cite{variationalAE} and a vanilla AE. They use PCA \cite{MACKIEWICZ1993303} for the two dimensional latent representation. However, also in their work they point out, that it is hard to interpret the informativeness of the data from one single visualization method. In \cite{2dlatent} the authors construct a VAE model with 2D latent space to investigate industrial time series data visually. Their aim is to ensure a higher interpretability of the data and thus to reduce the black box effect of deep neural networks. However, the interpretation of a two-dimensional latent representation from scratch is very intuitive since the degree of informativeness is not quantified. Also, the latent space dimension is an important hyperparameter and setting it a priori to $2D$ might cause loss of interpretability and informativeness, depending on the dataset. In a first step, this uncertainty can be reduced by using multiple visualization methods and applying dimension reduction methods to the latent space. A large class of examples for the increasing importance of latent and generative models as pattern recognition and representation algorithms can be found in the biological sciences. Examples constitute low dimensional latent representations of vocal communication signals \cite{birds, animalvocal, RePEc:plo:pcbi00:1008228}, protein structure \cite{Alam_reduced} or neural activity \cite{dabagia2020comparing,dimred}. In general, available attribute information is used as conditioning inputs without explicit supervision. Latent approaches can have advantages over hand labeling or supervised learning even for fields with a high amount of expert knowledge. However, a given latent representation depends on the interaction of the chosen Autoencoder architecture and the given data, the scalability of the latter, the visualization method and on whether the data forms a continuum or is clustered into subtypes. Thus a high amount of interpretability for the unsupervised case can only be attained with a sophisticated and inductive usage of Autoencoder architectures and visualization methods in combination.


\section{Pipeline and Methods}
\begin{figure}[h!]
\includegraphics[width=0.35\textwidth]{figures/pipeline2.png}
\centering

\caption{Our pipeline depicting the cross-correlative usage of clustering and embedding methods.}
\label{pipeline}
\end{figure}

Fig.~\ref{pipeline} depicts our pipeline involving a cross correlative usage of the clustering methods Optics (\cite{Ankerst}), DBSCAN (\cite{ester}) and k-Means (\cite{MacQueen}) as well as the embedding methods UMAP and TSNE. For the hyperparameter tuning of the involved Autoencoder models we used the Optuna (\cite{optuna}) package.

\subsection{Clustering Methods}
Given an input (at least topological, preferably metric) space $X$ consisting of a finite set of points $\{x_1,...,x_n\}$, the aim of clustering is to construct a non-nested flag complex of $X$. A \textit{clustering function} is any function $f$ that takes a distance function $d$ on a set of points $X$ and returns a partition $\Gamma$ of $X$. Herein, the choice of $d$ is crucial. 
\begin{itemize}
    \item K-Means takes a metric distance function and minimizes the distance of points to $k$ predefined cluster centers.
    \item DBSCAN is a density based method able to identify noise and clusters of all shapes. Its main hyperparameters are $\epsilon$, the critical value for which points are seen to belong to the same cluster, and $n_\text{samples}$, the minimum number of points that shall belong to one cluster. To measure $\epsilon$, the Euclidian distance is used. As $\epsilon$ is chosen globally, DBSCAN has its difficulties with clustering heterogeneous data~\cite{ester}.
    \item In contradistinction to DBSCAN, for OPTICS  $\epsilon$ is chosen from a dendrogrammatic graph called the \textit{reachability plot}. 
\end{itemize}
By construction, OPTICS thus falls into the regime of \textit{hierarchical clustering} methods - an idea to circumvent \textit{Kleinbergs impossibility theorem} (\cite{kleinberg}). Yet, in summary it can be said that each of the above mentioned clustering algorithms may have its drawbacks dependent on the respective data properties. However, in the unsupervised case, we don't know anything about the latter. We don't know how much 'main' clusters can be found within our data, and whether they are (globally) heterogeneous or homogeneous. Thus combining the above mentioned methods helps to investigate the input data space and use them for  maximal information gain. We started our analysis with OPTICS. As OPTICS and DBSCAN are closely related we subsequently used the already identified values of $\epsilon$ and $n_{\text{samples}}$ as starting points. This greatly reduced the meta-parameter space to be explored.  Lastly, we transferred the OPTICS limit to DBSCAN  and arrived at a fine tuned limit of clusters, that was subsequently used for the (embedded) k-Means analysis.




%\begin{figure}[h]
%    \includegraphics[width=0.25\textwidth]{figures/TSNE.png}
 %   \includegraphics[width=0.25\textwidth]{figures/TSNE_KM.png}
%    \includegraphics[width=0.22\textwidth]{figures/Kmeans_t-SNE_MNIST_cae.png}
%    \includegraphics[width=0.22\textwidth]{figures/Kmeans_t-SNE_MNIST_TAE.png}
%\end{figure}


\subsection{Embedding Methods}
As a visual analysis can only be performed with data up to 4 dimensions, low dimensional embeddings are crucial. In our work, we used TSNE and UMAP on a comparative basis.
\begin{itemize}
    \item The TSNE algorithm uses a distance metric (most commonly the Euclidian one) to mutually assign ''neighbourhood''-probabilites to the data points on which it successively minimizes the Kullback-Leibler divergence. Its most important hyperparameter is the \textit{perplexity} defining the minimum number of neighbourhood points. However, the hyperparameters used for the intrinsic optimization algorithm are also crucial for the final $2$- or $3$-dimensional embedding~\cite{tSNE, Hintons}.
    \item UMAP is an ansatz based on Spivaks \textit{fuzzy simplicial complexes} (\cite{Spivak2009METRICRO}) and by construction functorial. Yet, it crucially depends on the following metaparameters: First of all, the radius for the related C\v{e}ch complex, secondly, the metric and minimum number of neighboring points and lastly the metrified minimum point distance which also makes it prone to the curse of dimensionality~\cite{UMAP}.
\end{itemize}
UMAP produces an embedding by searching for a low dimensional projection of the data that has the closest possible equivalent fuzzy topological structure. Thus it expectably conveys more intuition about the entanglement structure of the data. We see this expectation confirmed in our studies concerning time series data. In contrast to this, TSNE appears more suitable for classification tasks.

\subsection{MNIST Test Data}
In fig.~\ref{fig:embeddings_tsne_mnist} we show the result for our test dataset MNIST \cite{inbookMNIST}. If our preceding OPTICS and DBSCAN analysis went well, we would expect the k-MEANS clustering to exhibit well separated clusters that - as for this case we can exactly track the labels to the latent space - also coincide with the digit clusters. On the left side of fig.~\ref{fig:embeddings_tsne_mnist} we can observe 11 more or less well separated clusters. Yet, the labelled right side shows that the 10 digit clusters are mingled quite a lot, such that the individual clusters do only partly match the digit clusters. This gets better when taking a convolutional instead of a vanilla Autoencoder, as shown within the second row, left in fig.~\ref{fig:embeddings_tsne_mnist}; and worse when using a topological Autoencoder (\cite{Moortopae}) as is shown in the second row, right in fig.~\ref{fig:embeddings_tsne_mnist}. The topological AE includes a loss term preserving connected components, which is not desirable for classification tasks including homologically similar data (such as the digit 0, 9 and 6). Transferring this strategy to the unsupervised task, we obtain information about the synergy of AE architecture and data via our visual analysis, for example the degree of topological (first-order) connectedness. Yet, this has still to be taken with a grain of salt: The dimension of the latent space is a sensitive hyperparameter. Choosing it too small might cause entanglements \cite{Eastwood2018AFF} between the significant data structures, that even become less resolvable when considering two dimensional embeddings (TSNE and UMAP). 



\begin{figure}[h]
    \centering
    \includegraphics[width=0.18\textwidth]{figures/TSNE_2.png}
    \includegraphics[width=0.18\textwidth]{figures/TSNE_KM_3.png}
    \includegraphics[width=0.17\textwidth]{figures/Kmeans_t-SNE_MNIST_cae.png}
    \includegraphics[width=0.17\textwidth]{figures/Kmeans_t-SNE_MNIST_TAE.png}
    \caption{ \textbf{Up Left:} Unlabelled; \textbf{Up Right:} ; \textbf{Down Left:} Labelled k-Means CAE; \textbf{Down Right:} Labelled k-MEANS TopAE. }
    \label{fig:embeddings_tsne_mnist}
\end{figure}

\section{Example Section}
In the following, we present a selection of data (Lorenz attractor, ECG5000 and DeepValve) that we investigated with the above mentioned methodology. 

\subsection{Lorenz Attractor}
As the processing of real world datasets is subject to epistemic and aleatoric uncertainty, we used the parameter-controllable Lorenz dataset to gain intuition about how this affects the feasibility of a visual analysis. We added uniform noise and Gaussian noise with three different standard deviations to our parameterized Lorenz attractor samples.

\subsubsection{Without Noise}

\begin{figure}[ht] 
  %\label{bla} 
  \begin{minipage}[b]{0.5\linewidth}
    \centering
    \includegraphics[width=.7\linewidth]{figures/labelled_UMAP_Lorenz_rho24.png} 
    \caption{$\rho=24$}
    \label{Umap1}
    \vspace{1ex}
    %\hspace{1ex}
    
  \end{minipage}%%
  \begin{minipage}[b]{0.5\linewidth}
    \centering
    \includegraphics[width=.7\linewidth]{figures/labelled_UMAP_Lorenz_rho99.png} 
    \caption{$\rho=99$}     
    \label{Umap2}
    \vspace{1ex}

  \end{minipage} 
  \begin{minipage}[b]{0.5\linewidth}
    \centering
    \includegraphics[width=.9\linewidth]{figures/3D_coloredPlot_Lorenz_rho24.png} 
    \caption{$\rho=24$}
    \label{2d1} 
    \vspace{1ex}
    
  \end{minipage}%% 
  \begin{minipage}[b]{0.5\linewidth}
    \centering
    \includegraphics[width=.9\linewidth]{figures/3D_coloredPlot_Lorenz_rho99.png} 
    \caption{$\rho=99$}
    \label{2d2} 
    \vspace{1ex}
    
  \end{minipage} 
  \caption{UMAP embeddings (upper row) and $2D$ projections (lower row) of a Lorenz attractor with $\rho=24$ and $\rho=99$ within an 8 dimensional vanilla AE latent space.}
\end{figure}


Fig.~\ref{Umap1} and fig.~\ref{Umap2} show the UMAP embedding of the input data space and 8 dimensional vanilla AE latent space of two Lorenz systems with significantly different dynamics. The red color in the color scheme denotes earlier times. Thus the small scale structures emerging at later times in fig.~\ref{2d1} are only visible in the latent space on the left, colorized in yellow to purple in fig.~\ref{2d2}. We can observe quasi-oscillatory behaviour about the collapsing point in purple. For $\rho=99$, we see a clear distinction of the butterfly wings in fig.~\ref{Umap2}, in comparison to the default teaser image. The wing structure within the latent space suggests that the system is not as stable anymore for $\rho=99$ as for the default parameters. 

\subsubsection{With Noise}

\begin{figure}[ht] 
  \label{fig:embeddings_umap_lorenz2} 
  \begin{minipage}[b]{0.5\linewidth}
    \centering
    \includegraphics[width=.9\linewidth]{3D_coloredPlot_Lorenz_default_noise.png} 
    %\caption{$\rho=24$} 
    \vspace{0.25ex}
    %\hspace{1ex}
    \label{Umap1}
  \end{minipage}%%
  \begin{minipage}[b]{0.5\linewidth}
    \centering
    \includegraphics[width=.7\linewidth]{figures/labelled_UMAP_Lorenz_default_noise.png} 
    %\caption{$\rho=99$} 
    \vspace{0.25ex}
    \label{Umap2}
  \end{minipage} 
  \begin{minipage}[b]{0.5\linewidth}
    \centering
    \includegraphics[width=.9\linewidth]{figures/3D_coloredPlot_Lorenz_default_noise_std5.png} 
    %\caption{$\rho=24$} 
    \vspace{0.25ex}
    \label{2d1}
  \end{minipage}%% 
  \begin{minipage}[b]{0.5\linewidth}
    \centering
    \includegraphics[width=.7\linewidth]{figures/labelled_UMAP_Lorenz_default_noise_std5.png} 
    %\caption{$\rho=99$} 
    \vspace{0.25ex}
    \label{2d2}
  \end{minipage} 
  \begin{minipage}[b]{0.5\linewidth}
    \centering
    \includegraphics[width=.9\linewidth]{figures/3D_coloredPlot_Lorenz_default_uniform_noise.png} 
    %\caption{$\rho=99$} 
    \vspace{0.25ex}
    \label{2d2}
  \end{minipage}
    \begin{minipage}[b]{0.5\linewidth}
    \centering
    \includegraphics[width=.7\linewidth]{figures/labelled_UMAP_Lorenz_default_uniform_noise.png} 
    %\caption{$\rho=99$} 
    \vspace{0.25ex}
    \label{2d2}
  \end{minipage}
  
  \caption{Two dimensional projection (left column) and UMAP embedding (right column) of default Lorenz attractor with Gaussian noise ($\mu=0$, $\sigma=1$) (upper row) and ($\mu=0$, $\sigma=5$) (center row) as well as uniform noise (bottom row) within an 8 dimensional vanilla AE latent space.}
\end{figure}


Fig.~\ref{fig:embeddings_umap_lorenz2} shows that not only the amount of expected noise, but also the underlying model belongs to the crucial prior assumptions. As one can see in the middle of fig.~\ref{fig:embeddings_umap_lorenz2}, spread out Gaussian noise with a standard deviation of $\sigma=5$ lets the butterfly wings disappear completely, but already for $\sigma=1$ the structure is much less resolved as for the noiseless case shown in fig.~\ref{fig:teaser2}. Uniform noise maintains comparatively most of the structure, as expected.

\subsection{ECG5000}
The ECG5000 dataset is a time-series dataset generated from an ECG. It consists of time-series samples with 140 timesteps each, all taken from a single congestive heart failure patient which can be assigned to one of 5 classes. Thus we expect 5 distinguishable clusters in our dataset. With our analysis pipeline, we identified a margin of $ 2 < n_{\text{clusters}} < 6$ by means of the vanilla AE architecture; and a margin of $ 3 < n_{\text{clusters}} < 6   $ by means of the topological AE. For time series, the usage of the topological AE is more generic than for classification tasks. Within time series, the data already exhibits a connection: through the temporal measurement sequence. Thus segments within this sequence naturally are associated to significant dynamical changes.
\begin{figure}[h]
    \centering
    \includegraphics[width=0.17\textwidth]{figures/OPTICS_t-SNE_ECG_LAE.png}
    \includegraphics[width=0.17\textwidth]{figures/OPTICS_UMAP_ECG_LAE.png}
    \includegraphics[width=0.17\textwidth]{figures/OPTICS_t-SNE_ECG.png}
    \includegraphics[width=0.17\textwidth]{figures/OPTICS_UMAP_ECG.png}
    \caption{Left: TSNE Embedding of the 8 dimensional latent space of a vanilla AE (upper row) and topological AE (bottom row). Right: Respective UMAP embedding.  }
    \label{ECG}
\end{figure}
Fig.~\ref{ECG} proves the increased expressiveness of the topological AE latent space in comparison to the vanilla one. Even the smallest cluster is clearly distinguishalbe within the TSNE as well as the UMAP embedding. However, one has to keep in mind that the ECG5000 dataset is already initially a relatively clearly structured dataset, whose clusters are not entangled. The difficulties that can arise when this is not the case will be discussed in the next section for the DeepValve dataset.

\subsection{DeepValve}
The DeepValve dataset is a time series dataset consisting of a sequence of random closing and opening events of an industrial valve. It contains 5 features in total. Already the projections of the 3 dimensional latent space embeddings for the vanilla and topological AE give an idea of its complexity.
\begin{figure}[h]
    \centering    
    \includegraphics[width=0.21\textwidth]{figures/3d-latent_space_DeepValve.png}
    \includegraphics[width=0.21\textwidth]{figures/3d-latent_space_DeepValve.png.png}

    \caption{Projection of the 3 dimensional latent space embedding of a vannilla AE (left) and a topological AE (right).  }
    \label{deepvalve}
\end{figure}
With our analysis pipeline, we could identify a margin of $4 < n_{\text{clusters}}<13$ for the vanilla AE and a margin of $3< n_{\text{clusters}}<6$ for the topological AE. Yet the visual post-analysis of the UMAP and TSNE embeddings with the true labels show that there are artefacts and the clusters are mingled, see fig.~\ref{deepvalve_lab}.
\begin{figure}[h!]
    \centering    
    \includegraphics[width=0.17\textwidth]{figures/labelled_t-SNE_DeepValve.png}
    \includegraphics[width=0.17\textwidth]{figures/labelled_UMAP_DeepValve.png}

    \caption{TSNE (left) and UMAP (right) embedding of the topological AE latent space for the DeepValve dataset with true labels.  }
    \label{deepvalve_lab}
\end{figure}


\section{Discussion and Outlook}
\label{discuss}
Our visual analysis pipeline helps to build intuition about the relation between input dynamics and latent dynamics. Furthermore, the process of assorting dataset types to suitable autoencoder architectures and clustering algorithms becomes more interpretable. However, we saw within our analysis of the DeepValve dataset, that the visual analysis can still be difficult even with our cross-correlative ansatz for real-world datasets. One source of obscurity is given by the possible remaining entanglement of the data within the latent space and thus also its $2D$ projection. When the latent space dimension is chosen too small, entanglements will be present and avert the separation of the substantial data structures. However, as is noted in \cite{Eastwood2018AFF}, despite the expanding literature on models which seek to learn disentangled representations, visual inspection remains the standard evaluation metric. A framework for the quantitative evaluation of disentangled representations when the ground-truth latent structure is available can be found in \cite{Eastwood2018AFF}. For real world data in the unsupervised learning case, this task is more cumbersome. One possible choice for a measure of entanglement or at least topological complexity could be neural persistence \cite{neuralpers}, a method coming from topological data analysis. It measures the generalization capabilities of deep neural networks via their structural complexity, which is supposed to increase layerwise over the whole network \cite{wheeler2021activation}. Neural persistence is determined via persistence diagrams \cite{persdig} over a weight-based filtration on the stratified data graph. As persistence diagrams are a comparatively easy to compute and visually interpertable tool, they constitute a promising complement to our pipeline. However, the computational experiments required for this would go beyond the scope of the given work. 















%% if specified like this the section will be committed in review mode
\acknowledgments{
The authors wish to thank the IAV GmbH for promoting and supporting this project.}

%\bibliographystyle{abbrv}
\bibliographystyle{abbrv-doi}
%\bibliographystyle{abbrv-doi-narrow}
%\bibliographystyle{abbrv-doi-hyperref}
%\bibliographystyle{abbrv-doi-hyperref-narrow}

\bibliography{template}
\end{document}
