There have been some headlines lately about Open Source AI Models, so let's clarify the difference between open-source and closed-source Large Language Models (LLMs).
Currently, popular LLMs like ChatGPT, Claude, Gemini, etc. are largely commercial models created by big companies. These models are regularly updated, have advanced features, and a lot of resources invested into them. However, they are closed-source, which means they also come with several drawbacks:
Expensive: Closed-source models cost money both for professional plan (via app) and business use (via API calls). When it comes to using API calls, the cost increases the more it's used, this can be a significant barrier for individuals and businesses alike.
Black Box: The main difference between open and closed-source is the ability to see and modify the source code, and when it comes to LLMs, "the model weights", making it essentially a black box.
Privacy Concerns: Since we use closed-source models either via their apps or APIs, we send our data to their servers, where it is then processed and returned to us with results. This poses a problem if you have sensitive information that you don't want to share, such as health-related data (HIPAA), personal data (GDPR), or even business confidential information. When you share your sensitive data with third parties, you don't know where that data can pop up again in the future or how it will be used; theoretically, it can even be used to train future models or shared with others.
This is where open-source models come in. Open-source models are often on par with the frontier commercial models, yet they may not be updated as regularly or have the most recent advanced features as the closed-source counter-parts.
"I use ChatGPT for basic personal tasks, 'cause legal won't let me use it for work". This is a common response when I ask friends, family, and colleagues how they have supercharged their workflow with AI. The answer lies in fine-tuned, locally run open-source models.
Let's look at how open-source addresses the issues we face with closed-source models:
Cost-Effective: Open-source models are free to use and modify. While running larger models might still incur costs for computational resources, it's often significantly cheaper than paying for API calls to closed-source models like GPT-4o.
Transparency: We get to see the source code and model weights and are allowed to modify them. A great example is the release of Llama3, an open-source model by Meta (Facebook). Meta surprised everyone when they shared their source code with the world, actually giving others the ability to build their own models. Other notable open-source models include Mistral, known for its efficiency, and Deepseek Coder, which specializes in code generation and understanding.
Privacy: One of the benefits of open-source models is that many of the smaller versions can be run locally on your computer or server (depending on your hardware). When run locally, your sensitive data is processed on your own machine, if done correctly it doesn't need to travel over the internet, meaning it never needs to touch anyone else's cloud. This is hugely beneficial for use cases that require privacy and confidentiality.
Open-source models can effectively perform various tasks like text generation, coding, image creation, and more. They are constantly being updated to match the abilities of the popular commercial models, there are thousands of them, and again, they are FREE. Therefore, they can be a great tool for various use cases.
Other Benefits of Open Source Models: The ability to fine-tune and customize open-source models is a game-changer for many businesses and developers. You can train these models on your specific data, making them experts in your domain. This level of customization is often not possible with closed-source models.
It's important to note that while smaller open-source models can be run locally (offline) on many of our personal computers, their larger counterparts often require more computational power. This means that to use the bigger open source models, you might need to run them on external servers or cloud platforms like Groq, meaning you might lose that privacy advantage of running locally, but it will still be more cost-effective than using the closed-source APIs (ChatGPT).
It's worth noting that one intriguing aspect of these LLMs, both open and closed-source, is that the exact process from prompt to output remains somewhat mysterious. Even the creators don't fully understand all the intricacies of what's happening under the hood, and is therefore an ongoing topic of research and debate in the AI ethics community.
I personally find myself moving between open and closed-source LLMs depending on the task at hand, yet whenever possible I opt for the open-source route.
How I use open source models:
GUI and RAG implementation: I use the models I downloaded with ollama alongside OpenWebUI to have a UI and Retrieval-Augmented Generation (RAG) already built in. I drop my documents in there and it works!
Processing and organizing medical documents: I use open source models to analyze, understand, and organize various medical documents.
Software development: I incorporate the models I download via ollama into some of the software products I'm building, removing the need to use expensive APIs to talk to frontier models (but it really depends on the task).
Coding assistance: For coding in the projects mentioned above, I use various coding models downloaded via ollama and integrated into VSCode with the Continue extension. Why pay for GitHub Copilot when I can get something comparable for free? These open-source coding models can sometimes outperform their closed-source counterparts in specific programming tasks.
A few of the models I'm currently using (each the smallest version offered by ollama - so that it runs smoothly on my machine):
These are all models offered by ollama natively, but I want to mention that it's possible to use models that ollama doesn't support out of the box by using a quantized model in a GGUF format. It's possible to quantize yourself, but I often find the quantized version of the model I'm looking for on hugging face.
Conclusion and What to Choose:
In summary, open-source models offer cost-effectiveness, transparency, and privacy advantages over closed-source alternatives. They're constantly improving and can be customized to suit specific needs.
The choice between open and closed source often boils down to the nature of your project, the data (limitations) you're working with, and the resources you have at your disposal. While closed-source models offer top-of-the-line features and regular updates, open-source models provide the flexibility of privacy, customization, and local deployment or cost-effective cloud options.
If you're looking for a model that is consistently updated with the latest advancements and features, a closed source model is often the way to go. However, if customization, transparency, and potentially lower costs are paramount for your project, an open source model may be the better choice.
As I mentioned above, I often find myself jumping back and forth between open and closed source depending on the specific task I'm tackling.
As the field of AI continues to evolve rapidly, it's exciting to see the innovations coming from both open and closed-source models. Whether you're a developer, a business owner, or just an AI enthusiast, understanding the differences between these models can help you make informed decisions about which tools to use for your specific needs.
(In a future post I will explain how to use ollama)
Comentários