Tejas Kumar

Build a RAG App In Under 20 Minutes

RAG++ London 2024 / 13:03

Watch this talk on YouTube

Transcript

18 paragraphs

This is an automatic transcript of the recording above. It is published in full and unedited, apart from correcting names the recogniser reliably mishears. It will contain mistakes.

00:00While that comes on, I'm Tejas Kumar. I'm one of the developer relations engineers at DataStax, and I'm super happy to show you Langflow today in production, what it would take for you to build an application using DataStax technology. Before we get into it, I'd love to just get a feel of the room. How many of you are developers here? Whoa, look at that. Take a photo of that. Incredible. Almost all of you. And so we have a really engaging developer track shortly after this. There's people from TlDraw. There's going to be so much great tech, but without too much ceremony. Look, I have only two slides and I'm a fan of show don't tell. So we're just going to get into it. Let's look at what we can build in just 15 minutes with Langflow on data stacks.

00:44The story we're going to tell is this. We love Netflix. I am a huge fan of Netflix. I use Netflix all the time. Great engineering. What we're going to do, though, is look at how we might build a more natural search experience for movies. I want to set the scene here. So imagine this. It's the weekend. You're sitting with someone, maybe yourself, and you want to watch a movie. I don't think in exact movie keyword titles. Anyone else like me? I don't think of the exact name. I usually, I want to watch something, maybe a romantic comedy or maybe a super. I feel like a superhero movie. I think like this natural language. And so let's try that. So I'm going to come in here and type in romantic comedies in the UK, right?

01:28And this is what we see, unfortunately. Yeah. He's like, ah, it's kind of sad. In fact, I've probably never seen this page unless I search like a human being. And it's interesting. The first suggestion here is what it says. Try different keywords. Because this is how search has been on the web so far. We match on keywords, but with the advent of gen AI, this is moving more towards natural language using semantic search and RAG techniques. And with Langflow, this is something that has become trivial to build. I'd love to show that to you. So Langflow is an application that you can use to declaratively and visually build end to end gen AI apps. It's a low code tool. What we're going to do is we've downloaded the data dump from a movie database and we're going to upload it into Astra DB.

02:17That's our vector search engine. And then we're going to build a GenAI app on top of that. So what I have here is a little bit of code. Now you can do this with low code, with Langflow, but we're developers as we saw. So let's look at some code. This is using our partner, Unstructured and what Unstructured will do. And I've chosen Python. But you can do this, there's many ways to do this. So what we're doing is we have a file called movies JSON. It's a big data dump of movies. We instantiate our Astra DB client and for each movie we use the Unstructured URL loader to go get the movie's details from themoviedb.org. Finally, we scrub the page content because what unstructured is going to do, it's a great way, by the way, to get your data rag ready, AI ready.

03:00So what unstructured is going to do is it's literally going to go to each web page and crawl the text content. Then we sanitize it. So we remove the login links, the login buttons, etcetera. Finally, we insert it into Astra db using this update one function. And look at this. This field vectorize is the text content of the page and this on the fly will be converted into a vector embedding. Anyone familiar with vector embeddings here? Yeah, all of you. Great. So that's one step straight from text to vector. Okay, I'm going to run this right now and we're going to actually ingest our movie data. So as you can see, this is happening in real time and this is going straight to astra Db one by one.

03:41Obviously we're not going to sit here and watch progress bars the rest of the day. I did this before. So let's go take a look at our Astra db database. This is it. This is our database. And as you can see, there's a ton and ton of movies here. And each movie has been vectorized into these embeddings that we truncate because we don't want to occupy a lot of screen. The cool thing is this is now ready to query over any API. Astra DB has a ton of SDKs and a ton of languages. You can already query this yourself without Langflow if you want to. We give you access to the data layer if your team needs just that, we also can do vector search with plain text. I could search for superhero movie right here in English.

04:22And this is going to be vectorized on the fly because we are using OpenAI. Excuse me here. And in just a few seconds what's going to happen is we get the Avengers, Justice League, Captain America, just beautiful semantic search. And you can also see the delta between various similarity results. This exists. You can build applications with this right now. Okay, Langflow takes this further. It doesn't just stop at the data layer, but you get the visual workflow builder. And that's what I'd love to show you. In fact, we're going to build an actual RAG pipeline from scratch with Langflow in just a few minutes. Okay, so I'm going to load up Langflow here and it's empty. As you can see, there's nothing.

05:01But we're going to fix that. I hope you can see the text. If not, I'll just zoom in. So every AI app starts with an input. And here we're going to use a chat input. So I'll just grab that. This is my chat input. It has nothing. What else do we need for a rag application? Well, we need an embeddings model. Anyone familiar? Everyone who's familiar with embeddings models here. Okay, good. Most of you, for those who aren't, it's a specialized machine learning model that can convert natural language into numerical representations for vector similarity comparisons. Okay, so we need an embeddings model again, that's right here in Langflow. So we come to embeddings and I'm going to choose OpenAI. But as you can see, there's a ton of options here.

05:39So we'll do OpenAI embeddings and I'm going to choose the text three embedding large model. So I'm just going to, with this dropdown right here, text three embedding large, just like that. Great. Finally we need to search our database of movies from Astra that I just showed you. So what we're going to do is go to our vector store Astra DB right here and drag it in. And notice we're not writing a single line of code. We're just assembling components and connecting things. So now I connect my chat input to the search input and the embeddings model to the embedding component right here, I hope. Is that, can you see that clearly? Yes. Great. I'm seeing a lot of heads. Great. So now we're getting data from Astra DB.

06:21We're going to use this to do some prompt engineering for our LLM. And again, notice how much code I'm writing here. None. Okay, so what do we do now? We need to get the text out of Astra. So let's take a helper that's called parse data, and this will convert our records from Astra Db into a string. For our prompt, we'll connect the search results up to that. And finally we'll engineer a prompt. We'll take a prompt component. Again, I'm going to zoom in here so you can see that's a prompt. And a prompt can add variables. What I'm going to do here is write the most classic rag prompt you can imagine, given this context, and create a variable. Answer the question. And it's question, that's just a variable.

07:04And you can see here prompt variables. Done. And now what I can do is I can zoom in, I can connect the prompt to the. Hang on a sec. So I get the context from the database. I connect that to the context variable, and the question comes from the user. Finally, we feed this prompt to an LLM, so we can choose any number of LLMs we want. And the cool thing about Langflow is you can swap them out. So right now I'm going to use OpenAI, but at any point I want, I could use Azure OpenAI. If I so choose. I'm going to take this prompt, connect it to the input, and that's it. Finally we have a chat output that I'll just add here. Just like that. All right. And now we connect the text up. That's it. So this is an end to end RAG pipeline built with Langflow.

07:55You ready to test it? Two of you? Chet was right, this side's alive. You all need some work. Okay, so playground, let's go. Let's ask for show me movies with big monsters. Let's try that. And we'll hit enter. And now it's going to. Well, okay, I miss the demo effect. Fantastic. Let's go add my. But the cool thing is it tells you what messed up. So I've got to give it my database. Aha. My database is rag. The collection is movies. There we go. I think we're good now. Let's try that again. Show me movies with large monsters. And really a good tool is not one that doesn't break, but one that helps you fix it. So, my guy, this side is starting to wake up. All right, show me movies with large monsters.

08:48And here you go. This comes straight from our rag. Database. Indeed. We just ingested those movies. It's a little bit big though. And so what we can do is turn on JSON mode. And this is going somewhere very special. So if we click on OpenAI and go to the advanced options, we can actually turn on JSON mode and now force it to respond with JSON. So I'll send exactly that same prompt. Show me movies with large monsters without a question mark this time as JSON. And if I really wanted to, I could even enforce a schema here saying, have these fields, not those fields, and create a deterministic flow in the end. I'm going to get back a response, hopefully today. There we go. Look at this. It's actual JSON that I can then use to build my application.

09:32Isn't that cool? Finally, you're driving this whole event right here. Okay, finally I can use this in my application. Look, there's an API button again. How much code did I write here? Zero. So there's an application button, an API button where I can generate a token, and this endpoint will deterministically run that flow and give me my JSON response every single time. What can I do with this? Anything. Who said that? I can convert. I can plug it. Thanks. I can plug it into a front end. A mobile app. I could do anything. So we built what we want in data stacks as a proof of concept, as a demo to show you what we consider what I actually use now for my movie searches. We call that movies. This is a front end that we built that uses exclusively Langflow as the backend, using exactly that flow.

10:28Do you want to see it? I'll show you. Have a look. Thank you. It's still that site, seriously. Okay, so let's just go with one of these pre filled prompts. Actually, let's do romantic comedies set in the UK. Let's make it local and we'll hit enter. And so. Okay, cool. Like, it's giving me these movies that are indeed from my database, but it's sort of got this format as a chatbot. And if you listen to devore earlier, I fully agree. He's like, who enjoys working with chatbots? I certainly don't. So I echo that and maybe we can do better, because again, Langflow is the API driving this. So using a great piece of software, we use the Vercel AI SDK in here. What we can do is a little bit better and we can say, what might this look like as ui?

11:17And I'm going to hit enter. And now look at this. Look at this, this is no longer a chatbot. We're actually generating. Thank you, rich user interface elements. Now, imagine you're an e commerce store and you want product recommendations. You can do this with Langflow on datasets. Generative UI is what we're calling this. Okay? And it's clickable, it's interactive. I will actually go to the details page, but it gets even better. I could say, show me the trailer for Notting Hill, maybe I want to preview before I commit. And now look at it. It actually shows me Hugh Grant right here. And we can actually just watch this together if you want. It's the actual trailer. And then I could be like, did people like Notting Hill?

12:05And it will switch back to text and give me the critical reception. As you can see, all of this one backend DataStax Langflow. How many lines of code? Zero. Absolutely. Hey, listen, that wasn't even 15 minutes. It was like twelve minutes. And so that's just a preview of what's possible. I want to thank you so much for coming out, and we really hope. Thank you, thank you, thank you, thank you. Production and transformative AI. We really hope you give it a try. We're gonna have a developer track in this room after the break at 04:00 p.m. we're gonna do a break now, so go outside, get some drinks, talk to people, anyone with a rhino shirt. We want to talk to you about DataStax. And then we're gonna do a developer track in here from 04:00 p.m.

12:49with some more great demos and talks. And there's an executive track over there. Pick your track. We're gonna pick up at 04:00 p.m. Thank you so much.

More talks

All 19 transcripts

Elsewhere

There is every talk I have given, all 69 of them, ConTejas Code, the podcast, and Fluent React, the O'Reilly book on how React works inside.