I have successfully defended my MSc thesis on the topic “Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer?”, advised by Yihong Liu (LMU Munich) and supervised by Prof. Dr. Vera Demberg (Saarland University) and Prof. Dr. Hinrich Schütze (LMU Munich).
Our paper investigates how architectural limitations of Transformers manifest after pretraining. Read it here.
We mechanistically explore the information flow in ICL tasks in Gemma model family. Read more here.
Our paper is a survey on current state of the art technologies for Turkic Central Asian languages. Read more here.
We study expressivity of SSMs in comparison to Transformers and RNNs. Read the full paper here.