Model Overview

Basic Information

• Model Name: CodeGen2-16B
• Developer/Creator:Salesforce AI Research
• Release Date: May 2023
• Version: 2.0 16B
• Model Type: Autoregressive Language Model

Description

Overview

CodeGen2-16B is a colossal language model developed by the visionaries at Salesforce AI Research. This behemoth of a model is designed to revolutionize the way we approach program synthesis, with the ability to generate and comprehend code across a vast array of programming languages.

Key Features

• Multi-turn program synthesis - a dance between the model and the developer, creating code together
• Infill sampling for code completion - filling in the blanks with precision and elegance
• Instruction tuning for following code generation instructions - a model that listens and learns

Intended Use

CodeGen2-16B is a Swiss Army knife for developers, designed to assist in writing and understanding code. From code generation to code completion, this model is a greate AI tool for those who seek to harness the power of AI in their coding endeavors.

Language Support

Supported languages (and frameworks) are as follows: C, C++, C-Sharp, Dart, Go, Java, Javascript, Kotlin, Lua, PHP, Python, Ruby, Rust, Scala, Shell, SQL, Swift, Typescript, and Vue.

Technical Details

Architecture

CodeGen2-16B is a Transformer-based model, with a staggering 16 billion parameters. It's amongst the smaller models, capable of processing and generating code with lightning speed, thanks to techniques like Flash Attention.

Training Data

This model is trained on the stricter permissive subset of the deduplicated version of the Stack dataset (v1.1)

Knowledge Cutoff

The model's knowledge is as current as the training data itself, up to June 2022.

Diversity and Bias

The training data is a melting pot of programming languages and domains, but the exact diversity and potential biases are not something we can discuss openly. It's a topic that requires careful consideration and research.

Usage

API Usage Example

License Type

The model is a gift to the research community, available for research and non-commercial use under the Salesforce AI Research license.