Model Overview

Basic Information

• Model Name: Nous-Hermes Llama2
• Developer/Creator: Nous Research
• Release Date: May 2023
• Version: 70B
• Model Type: Large language model (LLM)

Description

Overview

The Nous-Hermes-Llama2-70b is a 70 billion parameter language model developed by Nous Research. It is based on the LLaMA and Llama-2 architectures and aims to offer strong performance across various natural language processing tasks.

Key Features

• Long-form responses with a low rate of hallucinations
• Trained on synthetic GPT-4 outputs and datasets like GPTeacher, roleplay, code instruct, Nous Instruct & PDACTL
• Incorporates subject-specific datasets from Camel-AI and Airoboros

Intended Use

Nous-Hermes-Llama2-70b is designed for general-purpose language tasks, with potential applications in areas like instruction following, task automation, data analysis, and text generation. Its low hallucination rate makes it suitable for use cases requiring factual accuracy.

Language Support

The model supports English, German, Spanish, and French along with other languages.

Technical Details

Architecture

Nous-Hermes-Llama2-70b uses the AutoModelForCausalLM architecture from the Hugging Face transformers library. It follows the LLaMA and Llama-2 model architectures, which are based on the standard Transformer decoder.

Training Data

The model was trained on a diverse dataset, including:

• Synthetic GPT-4 outputs
• GPTeacher, roleplay, code instruct, Nous Instruct & PDACTL datasets
• Subject-specific datasets from Camel-AI and Airoboros

The training data covers a wide range of topics and genres, allowing the model to develop broad knowledge and language understanding.

Data Source and Size

The exact size and composition of the training data are not publicly disclosed. However, the model's 70 billion parameters suggest a very large training dataset.

Knowledge Cutoff

The model's knowledge cutoff date is not specified. As a recently released model, it likely has knowledge up to early 2023.

Diversity and Bias

Nous Research has not released information about the diversity of the training data or potential biases in the model. However, the inclusion of datasets from Camel-AI and Airoboros, which focus on diverse topics and perspectives, suggests efforts to mitigate bias.

Performance Metrics

Nous-Hermes-Llama2-70b has received positive feedback from users, who praise its coherence and low hallucination rate. However, specific performance metrics have not been publicly reported.

Comparison to Other Models

The model's performance has not been directly compared to other large language models in published benchmarks. However, its 70 billion parameters place it in the same class as models like GPT-3 and PaLM.

Usage

API Usage Example

Ethical Guidelines

Nous Research has not published specific ethical guidelines for the use of Nous-Hermes-Llama2-70b. However, the company emphasizes responsible AI development and the importance of transparency and accountability.

License Type

The model is licensed under the MIT license, allowing for commercial and non-commercial use with attribution.