70B parameter LLM with low hallucinations, based on Llama-2.
The Nous-Hermes-Llama2-70b is a 70 billion parameter language model developed by Nous Research. It is based on the LLaMA and Llama-2 architectures and aims to offer strong performance across various natural language processing tasks.
Nous-Hermes-Llama2-70b is designed for general-purpose language tasks, with potential applications in areas like instruction following, task automation, data analysis, and text generation. Its low hallucination rate makes it suitable for use cases requiring factual accuracy.
The model supports English, German, Spanish, and French along with other languages.
Nous-Hermes-Llama2-70b uses the AutoModelForCausalLM architecture from the Hugging Face transformers library. It follows the LLaMA and Llama-2 model architectures, which are based on the standard Transformer decoder.
The model was trained on a diverse dataset, including:
The training data covers a wide range of topics and genres, allowing the model to develop broad knowledge and language understanding.
The exact size and composition of the training data are not publicly disclosed. However, the model's 70 billion parameters suggest a very large training dataset.
The model's knowledge cutoff date is not specified. As a recently released model, it likely has knowledge up to early 2023.
Nous Research has not released information about the diversity of the training data or potential biases in the model. However, the inclusion of datasets from Camel-AI and Airoboros, which focus on diverse topics and perspectives, suggests efforts to mitigate bias.
Nous-Hermes-Llama2-70b has received positive feedback from users, who praise its coherence and low hallucination rate. However, specific performance metrics have not been publicly reported.
The model's performance has not been directly compared to other large language models in published benchmarks. However, its 70 billion parameters place it in the same class as models like GPT-3 and PaLM.
Nous Research has not published specific ethical guidelines for the use of Nous-Hermes-Llama2-70b. However, the company emphasizes responsible AI development and the importance of transparency and accountability.
The model is licensed under the MIT license, allowing for commercial and non-commercial use with attribution.