\paragraph{Simulation settings} We want to compare the performance of retrieving
the nodes blocks with missing edges (that are labeled as \texttt{NA} in the
incidence matrix).

For this purpose we generate collections of networks with the following
parameters:
\begin{align*}
    \bm{\pi}^m = \begin{cases}
                     \bm{\pi} = \left( 0.5, 0.3, 0.2 \right) & \text{for } iid                      \\
                     \sigma_1^m(\bm{\pi})                    & \text{for } \pi \text{ and } \pi\rho
                 \end{cases}, &
    \bm{\rho}^m =
    \begin{cases}
        \bm{\rho}  = \left( 0.5, 0.3, 0.2 \right) & \text{for } iid                        \\
        \sigma_2^m(\bm{\rho})                     & \text{for } \rho \text{ and } \pi\rho,
    \end{cases}
\end{align*}
for the block proportions, and two different structures with the corresponding
$\bm{\alpha}$,
\begin{align*}
    \bm{\alpha}^{modular} = \begin{pmatrix}
                                0.9  & 0.05 & 0.05 \\
                                0.05 & 0.2  & 0.05 \\
                                0.05 & 0.05 & 0.8
                            \end{pmatrix}, &
    ~\bm{\alpha}^{nested} = \begin{pmatrix}
                                0.9  & 0.65 & 0.1  \\
                                0.35 & 0.15 & 0.05 \\
                                0.1  & 0.05 & 0.05
                            \end{pmatrix},
\end{align*}

where $\bm{\alpha}^{modular}$ represents networks where there are look-a-like
communities, which tends to interact preferentially within the community and
less with the other communities. And $\bm{\alpha}^{nested}$ represents a common
structure detected in ecology with generalist and specialist species and a
\enquote{nested} structure.

The collections contain two networks ($M=2$) of size $n^{m=1}_1 =
    n^{m=1}_2 = 20$ and
$n^{m=2}_1 = n^{m=2}_2 = 120$. One collection is generated for each colBiSBM
model. And the nodes block memberships (i.e., the row and column blocks they
belong to) are saved.

Per colBiSBM model, 10 collections are generated and their results are
averaged.

In the network $m=1$ (i.e., the smaller one) a proportion of the edges
$p_{\texttt{NA}}$ see their values replaced by \texttt{NA}s, the
\enquote{forgotten} values are stored.

\paragraph{Test procedure} A LBM is fitted on the first network, and the
predicted block memberships are saved, along with the predicted links using the
inferred parameters. This will serve as a baseline to see if the use of the
collection benefits the predictions.

A colBiSBM model is then fitted (with a model matching the dataset considered)
and we store the same predictions.

\paragraph{Quality metrics} To benchmark the performance we use the
\emph{Area Under the Curve} (AUC) for predicted versus real link values and the
ARI for predicted versus real block memberships.

\begin{figure}[H]
    \centering
    \includestandalone{tikz/simulations/na_robustness/ari-dim-model}
    \caption{ARI in function of $p_\texttt{NA}$, the proportion of missing links
        for various colBiSBM models and their LBM counterparts}
    \label{fig:ari-dim-plot-na}
\end{figure}

\begin{figure}[H]
    \centering
    \includestandalone{tikz/simulations/na_robustness/auc-model}
    \caption{AUC in function of $p_\texttt{NA}$, the proportion of missing links
        for various colBiSBM models and their LBM counterparts}
    \label{fig:auc-plot}
\end{figure}

\paragraph{Results}
Figures~\ref{fig:auc-plot} and~\ref{fig:ari-dim-plot-na} show a
box plots named \enquote{sep-$model$} that
corresponds to the results given by a LBM fitted on data generated with the
corresponding \emph{model}. We will compare the results for one model box plot
to the corresponding sep-model box plot, serving as a baseline.

% TODO the ARI interpretation
For the figure~\ref{fig:ari-dim-plot-na}, our models almost always do at least
as good as the sep counterpart. The $iid$ model is the only one for which the
sep performs better on the columns block memberships.
The nested structure seems to complexify the block membership attribution with
only ARI less than 0.75

For the figure~\ref{fig:auc-plot}, in almost all cases and for almost
all models the differences are not significant but our models seems to perform
marginally better and are only a few times under their LBM counterpart.
This indicates that information is transferred from the bigger network when estimating the parameters and predicting link values.

On the differences between nested and modular structures, the latter shows
a smaller variance in the AUC with our models predictions contained between
0.7 and 0.9. Whereas for the nested structure, $iid$ and $\pi$ models are
in quite similar value ranges with small variances but $\rho$ and
$\pi\rho$ present smaller values and larger variances.

An explanation for the cases in which our models return lower values than
expected could be to look for in our simulation parameters. They may, combined
with the $\rho$ model be a difficult case for the estimation.
As we currently do not have identifiability results this is just an
hypothesis.