We propose LLM-Roleplay, a fast and cheap method for generating human-chatbot interaction logs (dialogues), using large language models (LLMs). By embodying textually described personas, LLM-Roleplay generates diverse, multi-turn dialogues that replicate the nuances of real conversational exchanges between human and chatbot. We evaluate this approach by comparing natural human-chatbot dialogues across different sociodemographic groups with those produced by our method. The findings demonstrate that LLM-Roleplay effectively mimics human-chatbot interactions, achieving a high indistinguishability rate (44% for Mixtral 8x7B out of 50%) between generated and real dialogues.

Introducing CM3Leon - a powerful, retrieval-augmented, token-based, decoder-only multi-modal language model. It excels in generating and infilling both text and images. CM3Leon leverages the CM3 multi-modal architecture and showcases the significant advantages of scaling up and fine-tuning using diverse instruction-style data. It stands out as the first multi-modal model trained with a recipe adapted from text-only language models. Incorporating a large-scale retrieval-augmented pretraining stage and a multi-task supervised fine-tuning (SFT) stage, CM3Leon achieves state-of-the-art performance in text-to-image generation with 5 times less training compute than comparable methods, boasting a zero-shot MS-COCO FID of 4.88.

We present a BART-like model: BARTSmiles, pre-trained on molecular representations. We quantitatively show that when applied to the molecular domain, the BART objective learns representations that implicitly encode our downstream tasks of interest. BARTSmiles consistently outperforms other self-supervised representations across classification, regression, and generation tasks setting a new SOTA on 11 tasks. Lastly, we show that standard attribution interpretability methods, when applied to BARTSmiles, highlight certain substructures that chemists use to explain specific properties of molecules.

We provide systems for en-ru and en-de language pairs for the WMT20 Biomedical Machine Translation shared task. For the en-ru pair, our submissions achieve the best BLEU scores, with en→ru direction outperforming the other systems by a significant margin. We explain most of the improvements by our heavy data preprocessing pipeline which attempts to fix poorly aligned sentences in the parallel data.