Faraday - Chat with AI Characters Offline
Offline first, privacy focused desktop app for creating AI chatbot characters.
I was on the lookout for a simple way to get a AI chatbot running on my old laptop without too much fuss. I discovered Faraday and was blown away by how simple it is to set up and get running in just a few clicks.
Let's take a quick tour of Faraday and check out the options and features to see if it's a good fit for you.
What is Faraday?
Faraday is a desktop AI chat tool that prioritizes privacy and simplicity. Faraday is available for both Mac and Windows
Faraday Key Features
Faraday offers a user friendly experience right from the start. You can begin chatting with ease using a simple one click Desktop installer that requires no coding knowledge. It's designed to work seamlessly, even with GPU and Metal acceleration included, ensuring a smooth experience without any complicated setup.
Faraday is commitment to safeguarding your data. The AI models that drive Faraday are stored exclusively on your computer. This means that no one can tamper with how your AI characters behave, deny your access, or remove your data. Your information remains under your control.
Faraday is a desktop application that functions offline. Whether you're on a plane, train, or anywhere without an internet connection, you can still enjoy its capabilities. Your chat data is saved directly on your computer, never sent to a remote server, ensuring your conversations remain private. You can choose to share specific chats to assist in improving the AI models, but it's entirely optional, preserving your privacy.
Faraday Settings
On the settings page you have the option to add your name and photo. You can also view your system specs and other details. I noticed that it detected my AMD Radeon 610, but I decided not to enable GPU support initially. I wanted to see how the performance would be without it first.
Faraday Model Manager
When I first launched Faraday, I went straight to the Model Manager where I was shocked how easy it was to choose the right model for my laptop. You can see small badges with stars that even recommend models based on your system specs. This just adds to the simplicity of the app making it easier to get started. You can choose a recommended model or browse the complete collection that is curated from the popular Hugging Face LLM platform.
My laptop specs are decent but keep in mind I do not have a GPU which could greatly improve performance and speeds of response times. But you will see later how well this performed.
- CPU: Intel i7 10510U
- RAM: 32GB
I chose the Llama 2 - Luna AI 7B model then navigated to the Character Hub.
Faraday Character Hub
I was genuinely surprised by the variety of characters available. I'm quite adventurous, so I decided to create my own character right away. I was more interested in having an AI assistant that could provide straightforward responses rather than the entertaining responses created by premade characters.
You can browse the complete Faraday Character Hub to see what's available.
The Character Creator is loaded with a multitude of settings, giving you the freedom to finely adjust how you want your AI to respond and behave. These settings empower you to customize your AI's behavior to match your preferences, creating a truly personalized experience. You can tweak various aspects to make your AI assistant interact with you in a way that feels just right for your needs.
I selected the model at the top, entered my name, and gave my AI character a name. I didn't make any changes to the other settings. I left everything else as default and began chatting with it. This will give me an idea of how the responses will be before I start to fine tune the character settings.
When you enter the first message, it will initialize the new character with the chosen model. That can take a couple minutes but once that is done, it's smooth sailing!
I know you are probably wondering how the performance is so I whipped up this quick gif to show you the speed of the responses.
Pretty good, right? The speed you're seeing in the image above is quite comparable to what I usually experience with third party AI apps like ChatGPT, which genuinely impresses me!
What's intriguing is that it seems to have more impact on my CPU rather than my RAM. Here's a screenshot displaying the system resources Faraday utilizes while processing a response.
Is it too good to be true?
Why is Faraday free and how long will this last? Well, I did some digging and here's what I found.
Is Faraday Truly Free?
One of the primary questions users often have about Faraday is whether it's truly free. In simple terms, yes, it is. Faraday will always allow you to run inference locally at no cost. (according to lead developers in their Discord) This means you can enjoy its AI capabilities without any charges when using it on your own computer.
The Developers' Incentive and Revenue Model
Naturally, you might wonder what the incentive is for the developers and how they plan to sustain this offering. The answer lies in a forward thinking approach. Faraday aims to cater to diverse user needs, and to achieve this, they plan to introduce a cloud-hosted inference option. This option will be particularly appealing to users seeking enhanced performance and the ability to run larger AI models. However, it's essential to note that this cloud-hosted service will be an optional, paid feature.
Final Notes and Thoughts
Faraday is an incredibly innovative tool, and my initial experience with it was a lot of fun. The more I use it and customize my character model, the more impressed I become. It's fantastic to have this tool with me when I'm traveling without internet access. I get hours of entertainment just from chatting with AI. These tools never cease to amaze me and keep exceeding my expectations.
While the devs contribute to open source libraries used within Faraday, the full-stack codebase for our Desktop App is not open source.
Although Faraday isn't open source, I felt it was worth the review here at Noted. I understand that this might raise concerns among some of you, especially given their focus on privacy despite the closed source nature of the app.
You can see the full explanation on the Faraday Discord community FAQ.
