Hovhannes Tamoyan


Doctoral Researcher at
Technical University of Darmstadt


Prof. Iryna Gurevych, Prof. Eduard Hovy

working on

LLM Evaluation and Code Generation

Publications and Projects

Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning

Armen Aghajanyan*, Lili Yu*, Bowen Shi*, Ramakanth Pasunuru*, Hovhannes Tamoyan, Luke Zettlemoyer et al.

Introducing CM3Leon - a powerful, retrieval-augmented, token-based, decoder-only multi-modal language model. It excels in generating and infilling both text and images. CM3Leon leverages the CM3 multi-modal architecture and showcases the significant advantages of scaling up and fine-tuning using diverse instruction-style data. It stands out as the first multi-modal model trained with a recipe adapted from text-only language models. We present CM3Leon, a retrieval-augmented, tokenbased, decoder-only multi-modal language model capable of generating and infilling both text and images. It is the first multi-modal model trained with a recipe adapted from text-only language models, including a large-scale retrieval-augmented pretraining stage and a second multi-task supervised fine-tuning (SFT) stage. CM3Leon achieves state-of-the art performance in text-to-image generation with 5 times less training compute than comparable methods (zero-shot MS-COCO FID of 4.88).
Beyond Neural Scaling Laws with Efficient Scaling

Hovhannes Tamoyan*, Michihiro Yasunaga*, Armen Aghajanyan

Modern deep learning is now a scale-driven game, where larger models, empowered by increased compute and data, dominate benchmarks. Our goal is to provide possibly best performing model for a given comput. To this end we try to fundamentally improving scalability through three approaches: Retrieval, Boosting, and Diffusion Augmented Transformers.
BARTSmiles: Generative Masked Language Models for Molecular Representations

Hovhannes Tamoyan*, Gayane Chilingaryan*, Ani Tevosyan*, et al.

We provide a BART-like model: BARTSmiles, pre-trained on molecular representations. We quantitatively show that when applied to the molecular domain, the BART objective learns representations that implicitly encode our downstream tasks of interest. BARTSmiles consistently outperforms other self-supervised representations across classification, regression, and generation tasks setting a new SOTA on 11 tasks. Lastly, we show that standard attribution interpretability methods, when applied to BARTSmiles, highlight certain substructures that chemists use to explain specific properties of molecules.
YerevaNN’s Systems for WMT20 Biomedical Translation Task: The Effect of Fixing Misaligned Sentence Pairs

Karen Hambardzumyan, Hovhannes Tamoyan and Hrant Khachatrian

We provide systems for en-ru and en-de language pairs for the WMT20 Biomedical Machine Translation shared task. For the en- ru pair, our submissions achieve the best BLEU scores, with en→ru direction outperforming the other systems by a significant margin. We explain most of the improvements by our heavy data preprocessing pipeline which attempts to fix poorly aligned sentences in the parallel data.
Low Energy Electron Beam Emittance Measurement at AREAL Accelerator

Hovhannes Tamoyan, Gevorg Zanyan, and Hakop Davtyan

The beam emittance measurement at low energies is an important issue to optimize the facility performance and compensate the space charge effects. In this paper, we present the results of electron beam emittance measurement for AREAL beam energy of 2.5 MeV using the quadrupole scan technique.

Frameworks and Tools

🦁 UrarTU

Robust ML framework featuring an intuitive YAML-based configuration system, streamlined Slurm job submission, and a versatile architecture. This empowers users to seamlessly navigate the intricacies of machine learning pipelines without losing focus.

🗂️ OrganizeNoc

Extensive suite of extensions tailored for researchers. Enhances the exploration of academic fields by simplifying the addition, querying, and extraction of insights from research papers. The suite includes PaperNoc, NoteNoc, and FindingNoc, collectively streamlining the management of academic materials in Notion through features such as metadata extraction, PDF highlight integration, and AI-driven paper queries.

🐈‍⬛ tmynNLP

Comprehensive NLP pipeline with insightful abstractions, seamlessly streamlining experiment setup and execution while providing a robust solution for a diverse array of natural language processing tasks.