JetBrains Releases First Open AI Model for Coding
JetBrains, known for its popular app development tools, has unveiled its first “open” AI model for coding. Mellum, a code-generating model initially released for their software development suites, is now available on the AI development platform Hugging Face. Trained on over 4 trillion tokens and equipped with 4 billion parameters, Mellum is tailored for code completion tasks, such as suggesting code snippets based on context.
Parameters in AI models represent problem-solving abilities, while tokens are the data bits processed by the model. To put it into perspective, a million tokens equate to about 30,000 lines of code. JetBrains emphasizes that Mellum is designed for professional developer tools, AI-powered coding assistants, code understanding research, educational purposes, and fine-tuning experiments.
The company trained Mellum on a diverse range of data sets, including permissively licensed code from GitHub and English-language Wikipedia articles. The training process spanned approximately 20 days and utilized a cluster of 256 H200 Nvidia GPUs.
While Mellum requires fine-tuning before use, JetBrains has provided pre-trained models for Python. However, these models are intended for assessing potential capabilities, not for direct deployment in production environments.
As AI-generated code becomes more prevalent, security challenges emerge. A survey by developer security platform Synk revealed that over 50% of organizations encounter security issues with AI-produced code. JetBrains acknowledges that Mellum may inherit biases present in public codebases and warns that its code suggestions may not always be secure or free of vulnerabilities.
Looking ahead, JetBrains sees Mellum as a starting point rather than a final product. In a blog post, the company expressed a focus on sparking meaningful experiments, contributions, and collaborations within the developer community. By making Mellum open-source, JetBrains aims to drive innovation and enhance the coding experience for developers.
