Vicuna FastChat T5 (3B) API

Basic Information

Model Name: FastChat-T5 (3B)
Developer/Creator: LM-SYS (primarily Dacheng Li, Lianmin Zheng, and Hao Zhang)
Release Date: April 2023
Version: Current Version
Model Type: Text (Chatbot)

Key Features:

• Encoder-decoder transformer architecture
• Fine-tuned on 70,000 ShareGPT conversations
• Autoregressive response generation
• Optimized learning rate and warmup ratio for fine-tuning
• Licensed under Apache License 2.0

Intended Use:

FastChat-T5 is designed for commercial chatbot applications and research in natural language processing. It can be used for generating responses in conversational agents and other NLP tasks.

Language Support:

Supports English. Other languages may be supported but with reduced accuracy due to the training data being primarily in English.

Architecture:

FastChat-T5 is based on an encoder-decoder transformer architecture. The encoder processes the input text bidirectionally, creating hidden representations. The decoder then uses cross-attention to focus on these representations while generating the response autoregressively from a starting token.

Training Data:

• Source: Conversations collected from ShareGPT.com
• Diversity: Includes various types of conversational data, reflecting diverse scenarios and user interactions. However, it may contain biases inherent in the ShareGPT dataset.

Use cases

• lightweight personal and embedded chatbots for applications with limited compute
• AI assistants integrated into productivity apps, IDEs, and notepads
• customer support and FAQ automation with real-time response
• educational projects for learning LLM mechanics and T5 architecture
• low-cost serverless AI for offline utilities or batch processing
• research and development of instruction-following conversational AI

Capabilities

• Function calling
• Streaming
• Structured outputs