The paper based on my Master’s thesis findings in the mismatch between token-level cross-lingual alignment and downstream task performance has been accepted to the MRL Workshop. Additionally, at the HuDS lab, we explored the impact of structural priors on loss and model’s linguistic capabilities, published here and accepted to the BabyLM Workshop.
Good news! I joined HuDS lab at Universität Göttingen, led by Prof. Dr. Lisa Beinborn. I will work on sample-efficient strategies for cognitively plausible multilingual NLP. I am also advised by Prof. Dr. Alexander Ecker (Universität Göttingen) and Prof. Dr. Edoardo Ponti (University of Edinburgh). Yay, let’s do cool research!
I have successfully defended my MSc thesis on the topic “Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer?”, advised by Yihong Liu (LMU Munich) and supervised by Prof. Dr. Vera Demberg (Saarland University) and Prof. Dr. Hinrich Schütze (LMU Munich).
Our paper investigates how architectural limitations of Transformers manifest after pretraining. Read it here.
We mechanistically explore the information flow in ICL tasks in Gemma model family. Read more here.
Our paper is a survey on current state of the art technologies for Turkic Central Asian languages. Read more here.
We study expressivity of SSMs in comparison to Transformers and RNNs. Read the full paper here.