polarolouis/rapport-CEI-MIA-2023
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Translated and added algorithms

Browse source
This commit is contained in:
Louis Lacoste 2023-06-15 10:39:02 +02:00
parent 4025c259f4
commit 447389301c
2 changed files with 106 additions and 30 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

BIN
rapport.pdf
View file

Binary file not shown.

136
rapport.tex
View file

@ -29,6 +29,10 @@
\chapter{Présentation de l'UMR}
\chapter{Context}
Develop on what is the SBM, colSBM, etc.
\chapter{Adjustment of colSBM to the bipartite case: colBiSBM}
\section{Variational Expectation step}
@ -51,11 +55,13 @@ Fixed point formula for the Bernoulli distribution:
\bm{\tau}^{m,1} \log(\bm{1} - \alpha) \]
\end{itemize}
avec $\text{Mask}^{m}$ la matrice qui contient des $0$ si la valeur est un NA et
des $1$ sinon.
with $\text{Mask}^{m}$ the matrix containing $0$ if the value is a NA and a 1
otherwise.
\section{M step of the algorithm}
Incorporate the equations from \cite{chabert-liddellLearningCommonStructures2023}
\section{Computation of the variational bound}
\section{Penalties}
@ -72,8 +78,8 @@ For the \textit{iid-colBiSBM} the penalties were modified in the following way :
avec
\[ N_M = \sum_{m = 1}^{M} n_{r}^{(m)} \times n_{c}^{(m)} \]
\end{itemize}
And thus the $BIC-L$ formula is now:
\[ \text{$BIC-L$}(\bm{X},Q_1, Q_2) = \max_{\theta} \mathcal{J} (\mathcal{\hat{R}}, \bm{\theta})
And thus the $\text{BIC-L}$ formula is now:
\[ \text{BIC-L}(\bm{X},Q_1, Q_2) = \max_{\theta} \mathcal{J} (\mathcal{\hat{R}}, \bm{\theta})
- \frac{1}{2} [\text{pen}_{\pi}(Q_1) + \text{pen}_{\rho}(Q_2) + \text{pen}_{\alpha}(Q_1, Q_2)]\]
\paragraph*{\textit{$\rho\pi$-colBiSBM}}
@ -92,10 +98,10 @@ For the \textit{$\rho\pi$-colBiSBM} the penalties are the following:
\item Penalties for the $\alpha$s:
\[ \text{pen}_{\alpha}(Q_1, Q_2, S_1, S_2) = (\sum_{q=1}^{Q_1} \sum_{r=1}^{Q_2} \mathbb{1}_{(S_1)'S_2 > 0}) \log (N_M) \]
\end{itemize}
And the corresponding $BIC-L$ formula:
And the corresponding BIC-L formula:
\[
\begin{aligned}
\text{$BIC-L$}(\bm{X},Q_1, Q_2) =
\text{BIC-L}(\bm{X},Q_1, Q_2) =
\max_{S_1,S_2} [
& \max_{\theta_{S_1,S_2} \in \Theta_{S_1,S_2}} \mathcal{J}(\mathcal{\hat{R}},\theta_{S_1,S_2})\\
- \frac{1}{2} & (\text{pen}_{\pi}(Q_1, S_1) + \text{pen}_{\rho}(Q_2, S_2)\\
@ -109,9 +115,9 @@ In order to explorer the bi-dimensional latent space $(Q_1,Q_2)$
we use the following strategies.
\subsection{Model selection}
In the following steps the model selection consists of using the $BIC-L$
In the following steps the model selection consists of using the BIC-L
criterion to select the model. We choose among the proposed models the one that
maximizes the $BIC-L$
maximizes the BIC-L
\subsection{Initialization and pairing of the models}
First to combine the information from the $M$ networks we fit a collection model
@ -136,35 +142,59 @@ Using the previously fitted models for $Q = (1,2)$ and $Q = (2,1)$ we choose to
perform a greedy exploration to find a first mode.
Meaning that for a given $Q = (Q_1, Q_2)$ we will compute all the possible
memberships for the points $Q = (Q_1 + 1, Q_2)$ and $Q = (Q_1, Q_2 + 1)$, fit
the corresponding models and choose the one that maximizes the $BIC-L$ as the
memberships for the points $Q \in \{(Q_1 + 1, Q_2),(Q_1, Q_2 + 1),(Q_1 - 1, Q_2),
(Q_1, Q_2 - 1)\}$, fit
the corresponding models and choose the one that maximizes the BIC-L as the
next point from which to repeat the procedure. We repeat the procedure until the
$BIC-L$ stops increasing $3$ times in a row.
BIC-L stops increasing $2$ times in a row.
% \begin{algorithm}
% \caption{Greedy exploration of the latent space $Q_1$, $Q_2$}
% \label{alg:greedy_explotation}
% \textbf{Commencer} initialisation
% \begin{itemize}
% \item Pour chacun des $M$ réseaux, inférer les paramètre avec $Q_1 = q_{1,0}$ et $Q_2 = q_{2,0}$
% \item Apparier les clusterings obtenus en utilisant les probabilités marginales, afin de faire correspondre les étiquettes des clusters obtenus.
% \end{itemize}
% \textbf{tant que} le BICL du meilleur voisin sélectionné à chaque itération n'a pas augmenté durant 3 itérations consécutives, continuer :
% \begin{itemize}
% \item Calculer toutes les séparations possible de chacun des
% \end{itemize}
% \end{algorithm}
\begin{algorithm}[H]
\caption{Greedy Exploration for Mode Estimation}
\SetAlgoLined
\SetKwInOut{Input}{Input}
\SetKwInOut{Output}{Output}
\Input{Fitted models for $Q = (1,2)$ and $Q = (2,1)$}
\Output{Estimation of the mode using greedy exploration}
\BlankLine
Initialize $Q = (1,2)$ as the starting point
Initialize $\text{BIC-L}_{\text{max}}$ as the maximum achieved BIC-L value
Initialize $consecutive\_count$ as 0
\BlankLine
\While{$consecutive\_count < 2$}{
Compute possible memberships for $Q \in \{(Q_1 + 1, Q_2), (Q_1, Q_2 + 1), (Q_1 - 1, Q_2), (Q_1, Q_2 - 1)\}$\;
Fit models with the computed memberships
Choose the model with the maximum BIC-L as the next point
\BlankLine
\If{$\text{BIC-L} > \text{BIC-L}_{\text{max}}$}{
$\text{BIC-L}_{\text{max}} \leftarrow \text{BIC-L}$
$consecutive\_count \leftarrow 0$
}
\Else{
$consecutive\_count \leftarrow consecutive\_count + 1$
}
\BlankLine
$Q \leftarrow$ Next selected point
}
\BlankLine
\textbf{Output:} Estimation of the mode using greedy exploration
\end{algorithm}
When this first estimation of the $BIC-L$ mode has been find we apply the moving
When this first estimation of the BIC-L mode has been find we apply the moving
window on it.
\subsection{Moving window to update the block memberships and the $BIC-L$}
\subsection{Moving window to update the block memberships and the BIC-L}
The \emph{moving window} is used to update the block memberships on rows and
columns and fit new models with those changes.
To define the window, we use a center point and a \emph{depth}, giving us the
bottom left corner ($Q_{1,center} - depth, Q_{2,center} - depth$) and the top right corner of the
window ($Q_{1,center} + depth, Q_{2,center} + depth$). All the points in this square will be
updated and contribute to the update of the others.
This procedure is repeated until convergence of the $BIC-L$.
This procedure is repeated until convergence of the BIC-L.
The procedure consists of two alternating steps:
\begin{itemize}
@ -173,6 +203,52 @@ The procedure consists of two alternating steps:
\item the \emph{backward pass}: computing the possible merges to fit the current model.
\end{itemize}
\begin{algorithm}[H]
\caption{Moving Window Procedure}
\SetAlgoLined
\SetKwInOut{Input}{Input}
\SetKwInOut{Output}{Output}
\Input{Center point $(Q_{1,\text{center}}, Q_{2,\text{center}})$, depth}
\Output{Best model with maximum BIC-L in the window}
\BlankLine
Define bottom left corner $(Q_{1,\text{center}} - \text{depth}, Q_{2,\text{center}} - \text{depth})$\\
Define top right corner $(Q_{1,\text{center}} + \text{depth}, Q_{2,\text{center}} + \text{depth})$
\BlankLine
\While{not converged}{
\textbf{Forward pass:}
\For{$Q_1 \in \left[ Q_{1,\text{center}} - \text{depth} ; Q_{1,\text{center}} + \text{depth} \right]$}{
\For{$Q_2 \in \left[ Q_{2,\text{center}} - \text{depth}; Q_{2,\text{center}} + \text{depth} \right] $}{
Compute possible splits from predecessors $(Q_1 - 1, Q_2)$ and $(Q_1, Q_2 - 1)$
Fit models with the block membership changes
Compare and keep the best model based on BIC-L
}
}
\BlankLine
\textbf{Backward pass:}
\For{$Q_1 \in \left[ Q_{1,\text{center}} + \text{depth} ; Q_{1,\text{center}} - \text{depth} \right]$}{
\For{$Q_2 \in \left[ Q_{2,\text{center}} + \text{depth}; Q_{2,\text{center}} - \text{depth} \right] $}{
Compute possible merges from predecessors $(Q_1 + 1, Q_2)$ and $(Q_1, Q_2 + 1)$
Fit models with the block membership changes
Compare and keep the best model based on BIC-L
}
}
\BlankLine
Update the best model based on the maximum BIC-L
}
\BlankLine
\textbf{Output:} Best model with maximum BIC-L in the window
\end{algorithm}
\paragraph*{Forward pass} The forward pass consists for a model at $(Q_1, Q_2)$
to compute the possible splits from the block memberships of its "predecessors".
The predecessors are the point at the left $(Q_1 - 1, Q_2)$ and below
@ -180,7 +256,7 @@ $(Q_1, Q_2 - 1)$ the current model (if they exist). To update the current model,
we take its predecessors block memberships and try to split one of the blocks in
two. Then the current model is fitted using this clustering as a starting
clustering. Once all the possible splits are fitted, they are compared, keeping
the best, in the sense of the $BIC-L$. If a model was already present it is also
the best, in the sense of the BIC-L. If a model was already present it is also
compared and the best is chosen as the model for this round at $(Q_1, Q_2)$.\\
The procedure then repeats for the point at $(Q_1 + 1, Q_2)$ until it reaches
$(Q_{1,center} + depth, Q_2)$ from which it repeats from
@ -201,7 +277,7 @@ $(Q_1, Q_2 + 1)$ of the current model (if the predecessors exist). To update the
current model, we take its predecessors block memberships and try to merge two
blocks in one. Then the current model is fitted using this clustering as
a starting clustering. Once all the possible merges are fitted, they are
compared, keeping the best, in the sense of the $BIC-L$.
compared, keeping the best, in the sense of the BIC-L.
If a model was already present it is also
compared and the best is chosen as the model for this round at $(Q_1, Q_2)$.\\
The procedure then repeats for the point at $(Q_1 - 1, Q_2)$ until it reaches
@ -212,7 +288,7 @@ for ($Q_{1,center} - depth, Q_{2,center} - depth$).
$(Q_{1,center} + depth, Q_{2,center} + depth)$, we know it was initialized at
least by the forward pass, no special case here.\\
At the end of the moving window pass, the model of max $BIC-L$ is the new best
At the end of the moving window pass, the model of max BIC-L is the new best
fit and the procedure can repeat until convergence.
\section{Networks clustering}