Vicuna: An Open-Source Chatbot Comparable to ChatGPT and Google Bard

Vicuna is an open-source chatbot built to deliver ChatGPT-like quality without matching OpenAI’s closed model approach. The project centers on a 13 billion-parameter model fine-tuned from Meta’s LLaMA using conversation data drawn from ShareGPT (user prompts and ChatGPT responses). At the time of recording, the model weights weren’t publicly available due to unresolved legal concerns, but a web demo was live—allowing direct side-by-side testing against ChatGPT.

The most consequential claim behind Vicuna is its reported quality: the authors say it reaches “90% of ChatGPT” quality when judged by GPT-4. That evaluation method matters because it uses GPT-4 as a judge to score responses across helpfulness, relevance, accuracy, and detail. In the reported results, Vicuna generally outperforms other open alternatives such as Alpaca and LLaMA, landing close to Google’s Bard and slightly below ChatGPT in the GPT-4-based ranking. Alpaca appears noticeably weaker in these comparisons, while Vicuna is described as “overwhelmingly” better than earlier open-source efforts.

Training details reinforce the “practical open model” narrative. The fine-tuning uses 70,000 ShareGPT conversation examples, and the project provides code and scripts intended to reproduce the training setup. The transcript also highlights that training costs are relatively low compared with earlier large-scale efforts: the reported figure for Vicuna’s 13B fine-tuning is around $300, contrasted with higher costs cited for Alpaca and other baselines. The project also includes guidance for fine-tuning on 8 GPUs, including parameters like batch size, learning rate, epochs, and token counts.

A key nuance is how evaluation differs across tasks. GPT-4-based judging is described as consistent and capable of producing detailed scoring rationales, but it’s also said to struggle with coding and math assessments—an important limitation when interpreting “quality” scores.

The live demo offers qualitative checks that mirror the benchmark story. For a philosophical prompt (“What is the meaning of life?”), both Vicuna and ChatGPT produce coherent, thoughtful answers. When prompted to roleplay as Dwight Schrute, Vicuna’s response is portrayed as more personal and character-specific, while ChatGPT’s answer is more direct. For a more sensitive question (“Who is hotter, Angela or Pam?”), both systems apply safety-style constraints: Vicuna provides a non-crude comparison and avoids forcing a binary choice, while ChatGPT either refuses or responds with a restrained, non-objectifying framing.

Overall, Vicuna positions itself as one of the strongest open-source chatbots in its class—especially among models derived from LLaMA—while still falling short of ChatGPT in the GPT-4 scoring hierarchy. The missing weights at the time of testing limit experimentation, but the promised release and the availability of training code keep the project firmly in the open-model spotlight.