Misplaced Pages

DBRX

Screenshot of DBRX describing Misplaced Pages
Developer(s)	Mosaic ML and Databricks team
Initial release	March 27, 2024
Repository	https://github.com/databricks/dbrx
License	Databricks Open License
Website	https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm

DBRX is an open-sourced large language model (LLM) developed by Mosaic ML team at Databricks, released on March 27, 2024. It is a mixture-of-experts transformer model, with 132 billion parameters in total. 36 billion parameters (4 out of 16 experts) are active for each token. The released model comes in either a base foundation model version or an instruction-tuned variant.

At the time of its release, DRBX outperformed other prominent open-source models such as Meta's LLaMA 2, Mistral AI's Mixtral, and xAI's Grok, in several benchmarks ranging from language understanding, programming ability and mathematics.

It was trained for 2.5 months on 3,072 Nvidia H100s connected by 3.2 terabytes per second bandwidth (InfiniBand), for a training cost of $10m USD.

References

1. ^ "Introducing DBRX: A New State-of-the-Art Open LLM". Databricks. 2024-03-27. Retrieved 2024-03-28.
2. "New Databricks open source LLM targets custom development | TechTarget". Business Analytics. Retrieved 2024-03-28.
3. Ghoshal, Anirban (2024-03-27). "Databricks' open-source DBRX LLM beats Llama 2, Mixtral, and Grok". InfoWorld. Retrieved 2024-03-28.
4. ^ "A New Open Source LLM, DBRX Claims to be the Most Powerful – Here are the Scores". GIZMOCHINA. Mar 28, 2024.
5. Wiggers, Kyle (2024-03-27). "Databricks spent $10M on new DBRX generative AI model". TechCrunch. Retrieved 2024-03-29.
6. "Data and AI company DataBrix has launched a general-purpose large language model (LLM) DBRX that out." Maeil Business Newspaper. 2024-03-28. Retrieved 2024-03-28.
7. ^ Knight, Will. "Inside the Creation of the World's Most Powerful Open Source AI Model". Wired. ISSN 1059-1028. Retrieved 2024-03-28.

v t e Generative AI
Concepts	Autoencoder Deep learning Generative adversarial network Generative pre-trained transformer Large language model Neural network Prompt engineering RAG RLHF Self-supervised learning Transformer Variational autoencoder Vision transformer Word embedding
Models	Text Claude DBRX Gemini GPT 1 2 3 J ChatGPT 4 4o o1 o3 Grok Granite Llama Mistral Large PanGu-Σ Image Aurora DALL-E Firefly Flux Ideogram Midjourney Stable Diffusion Video Dream Machine Gen-3 Alpha Hailuo AI Kling Sora Veo Music Udio Suno AI
Companies	01.AI Alibaba Anthropic Baichuan DeepSeek ElevenLabs Google DeepMind Hugging Face Kuaishou Meta AI MiniMax Mistral AI Moonshot AI OpenAI Runway Stability AI Synthesia xAI Zhipu AI
Category

• Large language models