Weekly deep dives on AI agents, coding tools, and building with LLMs - delivered to your inbox.
Free forever. No spam.
Subscribe Free
New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.
In this video, we're thrilled to announce Firestarter, an open-source chatbot creation platform. Firestarter allows you to create chatbots using just a URL, by recursively crawling, indexing, and auto-chunking the website's content. It features dynamic endpoints, syntax highlighting, and multiple chatbot creation. Learn how to set it up with various tech stacks, including Vercel AI SDK, Firecrawl, and Upstash Search. We also cover how to disable chatbot creation for public access and how to configure different settings. By the end of the video, you'll know how to clone the repo, configure API keys, and get your chatbot up and running in minutes. Don't forget to star the repo, and stay tuned for more exciting projects! Try it out: https://tools.firecrawl.dev/firestarter Star the repo ⭐: https://github.com/mendableai/firestarter 00:00 Introduction to Firestarter 00:34 Demonstrating Firestarter's Capabilities 01:02 Setting Up and Using Firestarter 02:02 Technical Details and Customization 02:46 Authentication and Security Features 03:19 Getting Started with Firestarter 04:05 Final Steps and Configuration 04:56 Conclusion and Call to Action
--- type: transcript date: 2025-06-18 youtube_id: T0jKhPzkrOk --- # Transcript: Introducing Firestarter: The Open-Source Chatbot Creation Platform In this video, I'm super excited to announce Firestarter. Firestarter is an open- source chatbot creation platform. And what it allows you to do is just with a single URL, it's going to recursively crawl that website that you gave it. It's going to index and autochunk all of the different pages as well as create vector embeddings. It will automatically create a namespace for each crawl that you create of the different websites. So, as soon as you create it, you're also going to have an endpoint that you can access. Within here we see the dynamically created model string where we're going to be able to go and access this from whatever application that we're building. If I just demonstrate this here, of course I could say something like hi, it will respond back in a conversational manner. For instance, if I say how do I make a request to the scrape endpoint. What we'll see here are all of the different sources that we're leveraging within the context of this response. Within here, we have things like syntax highlighting and all of that. But in addition to the actual playground is we also have these dynamic endpoints that are set up and you're going to be able to set up multiple. Also, by the end of the video, I'll show you how you can pull this down as well as spin this up. It only takes about a minute to get started with all of this. If I copy the curl command here, and if I paste in my request into my terminal, and I say something like, how do I leverage the search endpoint? I'll send that in. And then just like that we get the response back that we could leverage in our own application or even if you want to leverage something like this within an agent architecture in something like Langraph or whatever framework that you prefer. And also what we have within here is we have an index of all of the different chatbots that we've created. And also what it will do is it's going to automatically create the reference for each different chatbot you've created within the index page of the application. Within here you can go into these other chatbots and ask questions. You can also go within here and grab that model string and again plug it within your application wherever you see fit. Now in one quick note with the indexes for this page in particular, it is set up in a way where it will leverage local storage but you will find the reference for how these are stored within the application if you do want to swap this out for whether it's Reddus or Postgress or whatever you want to actually store these. Now in terms of the text stack, what this is leveraging is the Verscell AI SDK. So you're going to be able to use basically any LLM that is out there. I have it demonstrated with an OpenAI LLM, but you could swap it out to something like Grock or Anthropic or really whatever you're interested in using, whether it's the Gemini models or what have you. Now, in terms of the crawling functionality, so we are going to be leveraging fire crawl to do the web scraping as well as the content aggregation. And then for the embeddings creation as well as the retrieval that search functionality, we're going to be leveraging upstarch which makes it super simple to create an index as well as also reference and retrieve all of that different information within our application. One thing that I did want to highlight that is set up within the application if you don't want to set up authentication is you do have the ability to disable the chatbot creation feature. Essentially, if you were to go to that homepage or to try and make a request to that route, if that flag is set to true, you won't be able to create chat bots, but you will be able to reference and leverage the ones that you've already created. That's just one thing that I wanted to call out is if you do deploy this, but you don't want random people creating it and you don't want to set up authentication, you can just go ahead and set this to true once you've created your chat bots and you should be good to go. Finally, if you want to get started, you can go ahead and clone this down. Just to demonstrate that here, I'll clone down the repo and then within here I'll create a env. And then once we have the ENV, we just need three different things. From there, we're going to grab our firecrawl API key. And then next, we can grab our API key from OpenAI. So it will be openai API key just like that. And we'll paste it in. And then finally, we can make a free account on Upstash. We can go over to search. And within here, we'll create a new database. It will just take a second to spin up. And then once we have that we can go ahead select our env and we can copy those credentials and then we can paste it in. Next from there we can go ahead and pmppm install everything. And then once everything's installed we can go ahead npm dev. And you can obviously use npm or bun or whatever you prefer. And then within here we can go ahead and click start. And what I should see here is it will go through and begin scraping the content behind the scenes. It will deal with all the splitting chunking creating all of the different vectors. And then here we go. Here is our chatbot. If I send in a request, I say tlddr on firecrol. I see the sources coming in here. I have the references. And if I go and I test this within my terminal here, I see that the request works. One quick thing that I did want to point out is there is a config file at the root of the project where you can go and update things like the model provider. I put a handful within here, but you can go ahead and add in more. Right now, there's Grock, OpenAI, as well as Anthropic, but feel free to use whatever you'd like. Also a ton of different configuration settings within here. Whether you want to change out the default values for crawling or update some of the parameters for when you make a request to the AI model or if you want to update some of the settings in terms of how we're actually leveraging the reggg implementation with up-ash reddus. You can go ahead and update all of that within here. If you do like these types of projects, go ahead and give a star to the repo. I definitely appreciate it. But otherwise, if you found this video useful, please comment, share, and subscribe. Otherwise, until the next
Technical content at the intersection of AI and development. Building with AI agents, Claude Code, and modern dev tools - then showing you exactly how it works.
