Podcast Episode 2: RockIQ Rapid Fire Q&A from the AI Summit | Ep 218

Description

This week's AI Summit drew more than 670 registrations for the Community's first look at AI Agents in Rock.On this episode of Rock Cast, special guest Daniel Hazelbaker, a member of the team who built Rockʼs AI features, debriefs the AI Summit with Jon in a rapid-fire Q&A pulled straight from the event.They kick off by answering the most-asked question of the event: what version will these features be available in? They also discuss major topics like security, model options, MCP connectors, voice agents and technical details.If you haven't watched the AI Summit, we recommend starting there first and then tuning back in for this episode. Don't forget to share this recording with your leadership to help them catch the vision for where Rock is heading.Visit the show notes to find all the resources talked about in this episode. Don't forget to join the new Rock Cast Rocket Chat Channel to see what other churches are saying about this episode.

Transcribed Content

Hi, guys. Welcome back to ROTcast. Thanks for joining us today. This is a special edition episode with our guest, Daniel Hazebaker, who's been an instrumental team member in developing the AI features. And today, we're gonna go over all the questions from the AI Summit. And if you haven't yet, it's on our YouTube. So make sure you watch it there first and then give this episode a listen. So, John, can you kick us off and tell us what version will we need for this to be available? I don't wanna say, but I will give some hints and we will have a firm answer in about two weeks. And I think we said that on the AI Forum. But I just want to give a little bit of clarity of what we're thinking. But do not hold us to these. We're not on record. This is not the official two week notice. Our hope is that we're going to get it into nineteen point four. So, 19.3 is going to be going to Alpha Beta testing here real soon. Our goal is nineteen point four. Some things could happen. Emergency patch might need to be put in for some other security issue or another performance thing. I think it's gonna be very uncommon and unlikely, but it could happen. What we're trying to track to is 19.4. And we're gonna have more guidance on when that might be and the exact version in two weeks. However, I'd say, my question back to anybody who asked this question or anybody who's thinking about this question is, are you on '19 yet? Don't wait until this is out in 1940 and then get to '19. you have a big hurdle to get to right now. You got to get to nineteen. You get through the update, you got to get through some announcements to your staff, some communications. So, if you're asking that question and you're not on 19, I would say, well, first, the critical path is gonna be you getting to 19. Yeah, that makes a lot of sense. And I'm glad we're starting off with that one because I feel that was the first question people started asking. It would've been my first question. So, Daniel, does the agent run permissions based on the logged in user or the agent's own permissions? All the permission checking is done against the logged in user that's using the agent, and that's also true when going in through MCP. It'll use, the the authenticated user going through MCP to do all the permission checks. Okay. Got it. John, can you tell us, when you ask for information about the the family, can family information include any known relationships? Okay. So it's a good question. So when you ask it anything, tell me about a person or tell me about a family. Inside the system prompt, you can give it little hints of what you really want to add to that. By default, it's going to just pull out some generic type things. But if there's a program that your church does that's really important and you always want it to be included in those types of things, you can hint, Hey, when I ask for details about a family, a person, include this. Now specifically what they're asking for is known relationships. And the answer to that is a little bit more difficult. So known relationships inside of Rock are basically a type of group and there's group members. These are tied to individuals, not families. So, if you wanted to include this, your system prompt would have to be pretty detailed to say, hey, whatever asked information about family, you need to go to each individual in the family, you need to go check what its known relationships are, see if they have those, that's going be a lot of tool calls. So the AI can do it, but that is going to be a very slow return. So a few things. One is, I would kind of really want to know more about what's the requirement behind the requirement or the requirement behind the question, because it's probably a better way. That said, we're only getting started on the creation of our skills and tools. And so even from this, we can probably take though we don't really know the why, we can probably go back and refactor a bit the GetPerson or GetPersonProfile tool call to add in the information about the known relationships so that additional calls aren't needed. So we're gonna take that feedback. I'd still would love to know what the actual use case is. Regardless though, if we put it on the person, which is the right place. If you could ask for about a family, it's gonna have to call that tool and number of times of how many family members you have. So it's gonna be a little slow. So again, I'd love to know more about it, but that's a long winded answer. Okay, got it. Daniel, back to you. When the email to the person is sent, what does the from line say? The from line is always gonna be from you at this point. The tool is really meant to be sending a personal communication, not a templated system email. So it'll always, point back to you. If the reply, it's gonna go to you as the the sender of that communication. Okay. And John, how is using an MCP connector different from giving Claude an API key? I this question because I've had this question so many times in my mind, and I keep having to, , re answer it myself because I forget the answer sometimes. And it's a good question because MCP and a regular API, they're kind of the same thing. MCP really is this kind of special use API. So why do we have it? Typically, with an MCP type style API, it's special tunes to the purpose. So we can kind of tune it and say, hey, need an I wanna have these types of calls and these types of calls are gonna return these types of information. A typical REST call is going to be fairly bland and fairly not very use case tailored. This is person, people. The MCP, you get a little bit more context. I'm searching for a person. Okay. Well, now I know what you're looking for. I'm going return back this information. But I think the most important thing is if we were to give Claude our API docs and an API key, we just gave it the API key. With MCP, it doesn't have access to the API key. In MCP, the user authenticates in the harness and through OAuth and the harness has the API key, but the AI model doesn't have the API key. If you did the other way, the AI model would have the API key and could potentially do some dangerous things with that. So, MCP definitely the way to go. It's a little bit nuanced, but I would say the biggest thing is it's more secure. Okay. That's good to know. Daniel, can the communication component be disabled? Yes. Everything in the the agent and MCP, is designed with security and customization in mind, you can turn on and off specific pieces. If you wanna let them draft the communication but not send it, you can turn on the tool to draft, but turn off the tool to send. Mhmm. So you can get very Sounds specific with the specific and and even decide who's allowed to use the send versus drafting. Yeah. That's great. So, John, back to security. I assume platform, it respects user roles, but the API is probably admin. Or can you assign roles to the API key? Yeah. This is probably a misconception. So everything that we showed both inside the Rock chat and MCP is always in the context of the person using it. There's no administrative, it has complete access and there's really no API key. There's an MCP you do authenticate through OAuth, but it's more of a little token that's not one time, but it's a short lived token that keeps getting regenerated for you. It's much more secure and everything is operating as the person who's using it. Okay. So, Daniel, can you tell me about permission based queries? For example, could I prevent users from deleting certain things through an AI chat? Absolutely. Again, kind of similar to the last answer, each action is its own tool. So we have a add or update group tool, and we have a delete group tool. So you can say , we don't want anybody to use the delete tools and just exclude them from the agent completely. Okay. So, John, will it be possible to connect Rock AI to a locally hosted model? So it depends. If you're going to the MCP route, absolutely. Because MCP, the AI model is not us. , we have no control over that. That's another harness, another agentic tool is doing all of that. That's the orchestrator that we talked about in the call. If you're using the in line in product chat, then it's our orchestrator, it's the model we choose. And today, you cannot use a local model. It's something we could possibly look at in the future. I think at this point, local models, there's a lot of people who want to use them and are interested in them. And that's not a bad thing. I don't think they're needed right now. I think they could be needed in the future. if things go off the tracks, that might be a path. But right now, I think if you want to innovate and innovate fast, that's that's a level of infrastructure and a level of complexity that's just going to slow you down. That again could change. The world is changing so quickly. I think it's interesting to keep your eye on that. But to actually start investing in that, I think it might be a little early to go down that path at this point, unless you're a hobbyist and you just want to do it in your own time. All right. So Daniel, if we only use MCP and Claude, do we still have to add the spend and Rock? You will not need to. MCP, as John said, is is kind of outside of our harness. So you'll have to pay Claude or whatever system you're using for the AI part. But to connect to the MCP, you won't need to set up or pay for anything through Spark. I love that. I just love the fact that it's just free. You don't have to and I say free not from money, but just free dumb. , you're not we're not trying to do to close you in. We just had to close you in when you when it costs us money. Yeah. But I love that fact that you could just use an MCP and you're , just do good stuff. Yeah. I think that's aligned with a lot of the heart behind what we build. Yeah. So as self hosted open source models keep improving, does the Rock team have plans to explore integrating one or even training their own model for their main tasks, for the main Rock task to pass cost savings onto the community and leave only the more complex requests for frontier models? That's a very long question. That was a mouthful. It's a good one. I it. So our plan when we release the onboarding experience, if you looked at the screenshot, you might have noticed it said standard in some of the text. Our plan is to provide two options. A standard option, which we recommend, and that's gonna be using a low cost frontier model. And then there's gonna be another option of economy. And that's going to use an open source model. We still have to pay for it because it's running on someone's infrastructure. It's a lot cheaper. The reason why we don't really recommend it is the quality. The agentic tool calling quality is going to be less and it'll probably be slower. But if you're trying to save costs, what I would recommend though is try running standard and seeing what the costs really are. When I'm looking at our spend, I'm , wow, that's really not that much. But results could vary based on some configuration and what your staff is doing. And remember, this is only the in chat, the in Rock in chat costs. If you're using MCP, that's someone else's bill and it's whatever you choose. But let's talk about open source and fine tuning. Again, I kind of go back to , should you run your own infrastructure or inference infrastructure? I would probably say no, unless you're a hobbyist right now. Again, that could change. I think this kind of same thing a little bit on the open source front. I would probably right now, I'm leaning towards wanting to get my work done and being productive and having high quality. I think very, very soon though, that's gonna pivot a little bit. And some of these open source models are gonna be things we wanna worry about with fine tuning. Just listening to a podcast on drive in about how Microsoft is doing that with some of their non frontier models that aren't very good, but they fine tune them and they actually have better results than even the best frontier models right now. Because they're fine tuning it for the specific use cases. I that saying that's get the low hanging fruit. This is high hanging fruit or mid hanging fruit. right now we're gathering the low hanging fruit. Spend a lot of time and a lot of effort and a lot of research and a lot of brain damage to get this far. We'll go up to higher in the tree as we get there. Just know though, that we are staying abreast of all this. And it's not something that we're not thinking about. Mhmm. Absolutely. So Daniel, where are the chats stored and is there a way for Rock admins to review them and run evals to calibrate agent performance rather than waiting and hoping for user feedback? Yeah. Definitely. The chats are all stored on each Rock instance in the database. There's a couple tables to get to all that, so they can run all the queries they want to figure out know, how their users are using it and even which users are using it or not. Okay. So can you tell me, is there a way to rate limit certain users in RockIQ? , I can see some staff over utilizing AI tools possibly unnecessarily and using up a lot of credits. There's no way to to prevent it ahead of time. There's no rate limit. Mhmm. But with those queries, they can can build some data views that might show them who are the top spenders, so to speak. Each chat records the number of tokens that were used for the chat and the session in each message so they can kinda get a feel for was that an expected spike because they're asking one particularly hard session, or is that user just has a lot of stuff going and they might need to be reined in a little bit? So the rate limit is per organization, not per person. You can put a rate limit at the organization where per month it can't go over a cap. Right. Yes. Yeah. But you can't do that per person. Yeah. There there's a spending limit per month. There's not a rate limit of they can only do so much per hour, but you can keep them from from overspending the entire budget. So you could create your own token maxing board. Yep. Not recommended. In fact, the whole industry is that was a two week thing where they put these boards up to see who's who's spending the most and it was kind of a it was kind of a reward system that you want to be at the top of the board. And then they realized that that was a dumb idea. It's a lot of wasted tokens. My goodness. Yeah. It's what what happens when you have too much money? Okay. So are you creating shared skills? And if so, do those live in Rock or in an enterprise or Team Cloud account? And does that mean there's , we'll pause there. I'll start breaking these up. Yeah. So this is really interesting. And this is where our words fail us. So we talk a lot about skills and tools. The whole, , session was about skills and tools. And that's exactly what they are. But those are in Rock. Those are Rock skills and Rock tools. I think what this person's talking about is clad skills inside of your harness. So you have skills on both sides. You have skills in the Rock side and you have skills in your harness. And that's good. I've actually been doing that on my hobby project. I have an MCP that can do things and I have and those are skills and I have skills inside a cloud that give me the business logic. So and I I think this is good. You wanna have both actually. And that was kind of a moment for me because I was , well, I have this MCP, I don't need this anymore. But what you want inside your harness skills, the Claude skills is some of the business logic that's unique to your church. So you might say something , hey, when you update this person's attribute, this attribute, consider these things or do these things. And so that means that when you go inside of MCP and say, Hey, I want to update this person's baptism date. It's first going to call the Claude skill and say, Oh, I'm updating a person. What are the rules here? They say, Hey, when you update baptism date, you also need to check this box. Ah, okay. So now I'm going to go call the skill inside of Rock, and I'm going to do both those things. So you want both. But then they continue on. So in my mind, which is very simple right this, we're just in the elementary stages. In my mind, what's in Claude is yours, and what's in Rock is mostly ours, and you can add some stuff too. But I don't think we need to ship anything that would go into the Claude's side. I think that's where the churches need to customize their own. Where they put that is up to them. , yeah, it's a good idea to put into a GitHub repository that's shared and in enterprise account. Yep. That's what we do internally. But how they do that? At this point, I don't see a need for us to to make those Claude skills. I think that's up to the church. But everything's changing. So tomorrow, I might disavow anything I just said. Okay. Yeah. Make sure you get that disclaimer in there. Yeah. So is creating and editing skills only limited to admins? Or is that for Well, again, I think they're talking about the Claude side skills. So that would be up to you guys. On the Rock side, yes, out the box that is true. You could make a new security role and let other people add skills if you'd . Just anything in Rock, the security is very permissive and customizable. But I would proceed with caution. I would not give many people access to do that. Good to know. So what's the typical size of a context window? Right now, most of the models that we're using and testing with our 128 k. That's actually a lot, even though a lot of newer, code developed models are touting a million. But those are very different use cases in what you're gonna be using this for in Rock. Mhmm. And again, that's that's also only applies to when you're using the built in UI inside Rock. If you're using MCP, you're on their rule set of of context window size. But right now, it's a 128 k. That will probably go up just as new models come out. They they tend to grow their context window size. But realistically, that's a that's a lot. You can do a lot in hundred and twenty eight k. Alright. So major advancement in AI over the last six months has been agentic coding. Rock's website structure isn't well suited to full AgenTic coding because of the page block and block setting structure. Is there a plan to create agents, skills, tools that would allow coding Rock sites using AI? This is definitely something that our team has been thinking a lot about. And I think there's a lot of different tracks of of how this could go. You definitely can and we're actually We have a team member prototyping some lava skills to create some of this. I don't want to go into too much detail because if I say it, then I'll very soon that's what we're doing. It's not, it's a research. So, we're creating internal skills using Lava to do some of this. That could work. There's also some new technologies coming about how you build websites in Rock. You'll always be able to do what you do today with the blocks and settings because that's gold. But we want to go out and make some other ways and that might be more agenticable. And there's also some stuff going of how do we just do complete vibe coding? , how can you vibe code something? I think at the end of day, in my mind, you're still gonna want Rock's page structure for the header, footer, the frame. And then what you put inside the frame is what you vibe code. I can't really I can see a lot of danger if you let it vibe code the whole thing. At that point, would say, okay, well, don't do Rock. Just go vibe code your own thing. Use Rock API. I think that would be a mistake. I think you're gonna have to rewrite a lot of stuff and you're gonna miss out on a lot of business logic. I would kind of hang in there. I would say today though, if you just put an HTML block on a page, you could easily write a skill that says, just update that HTML block and invite that. But to do this, I think a lot of times we go straight to the skills that. But I think what and what we are working on is a lot of the supporting information. So say you could do all of this. You still have to understand what are the helper classes, CSS classes? What is the markup? What's the standard Rock markup? Because yeah, I could go put anything in there, but you wouldn't want to look strangely different than the rest of your website. So we're working to pull all of that information together. And we internally, we have some of those skills that can markup Rock. And I'd say they're doing really well, but they're not perfect. And so we're fine tuning that, I suppose you could call it. And then we need skills that understand lava and they can write really good lava. the AI today does okay, but it tends to also not understand perfectly all the different capabilities of lava versus liquid. And so we need to keep documenting that and adding more context. Yeah, we did that really good Rockhurst episode previously on vibe coding. We talked about the risks and the benefits. Mhmm. That was we'll probably link that in the show notes. So, Daniel, you mentioned, organizational prompt instructions that ref referenced output format an HTML table. How does that work when the same agent, skill, or tool is used across web and mobile context? Is there a way to define platform specific output formatting? Yeah. Well, first, right now, the AI agent only works on web. There's no mobile piece yet. There's a mobile voice agent piece, but that kind of goes through a different harness system altogether. As far as the HTML, what's actually being rendered is markdown from the language model. And then in the UI, we're converting that markdown to the HTML table. So in the future, the markdown should be compatible with, , other systems mobile or whatever. When and if we get to that, we would also add a way to to kinda know ahead of time, , oh, is this being used for web or some other platform so you can kinda tweak your system prompt accordingly? Because you might on mobile, might wanna have it be even more concise than it would be on web. Okay. So do these agents perform caching to minimize the cost of submitting similar prompts? Today, no. We're very familiar with that concept. I think the question we're dealing with is first is, we got to get this out. , let's get out phase one of this release version one. There's a concept of just ship, we just got to ship this. We are looking at that. I think it's just a matter of when and how to cache. Realize that we are not locked into a model here. We have not said, Okay, we're just doing Claude. The model selection, which we're going talk about a little bit later on the podcast here is going be a little bit different. So, we have to make sure whatever we do goes across models. We don't lock ourselves into one vendor. And then we have to determine what should be cached. I think some of the tool data results could be cached, but you wouldn't wanna cache a lot of the user stuff. Well, it might be similar. It's gonna be radically different in terms of intent. So we we need to kind of, , strategize on what gets cached and when it gets cached. Okay. I think we only have a few more questions here, but this has been great to rapid fire all of these. So Daniel, can you tell me what kind of increase in system resource usage, CPU memory, and so on can we expect on the machine running Rock once IQ is added? Very little. When you're using the agent, most of the processing is is in the agent, which is simply up in the cloud somewhere. If you're writing custom tools that run very complex SQL queries, that's always gonna put some load on your server. But your typical, your typical, interaction with the agent of, hey, look at Ted Decker, tell me about him and his family. That will probably actually use less resources than going to his profile page would use in turn. That's a really good point. I mean, there's a lot it's a lot more efficient. Yeah. Okay. Why do a lot of those tools use SQL? I don't know if I said that right. Yep. In Lava instead of entity commands. Well, when they're referring to that, I think they're referring to the examples we used in our in our call, the AI Forum call. We were simply that the purpose of those examples was to give a very basic example. For instance, one of the tools showed group types and so we just listed the group types. If it was me and I was going do it for real in Lava, I would have gotten that out of the cache. I'd gotten the group types out of cache. I would have massaged them to not bring back, not to just dump all of the properties of the group types, but just the few that we needed, which in the case we did, it was, I think, to see ID and the name, I might potentially have added description. Then I would have a whole bunch of other stuff to massage it and do it right. But that lava would have been Run the is authorized checks. Yeah, I mean, that would have been pretty detailed. And we're just we're not trying to scare people. We're just trying to give simple, simple, simple examples. So, a lot of cases, perhaps, I would pull from cache or maybe use entity commands. I do think in some cases, the SQL is still nice. But yeah, there's a lot of rubrics that have to go through your brain on what you should use that. Is this going to change? Is there business logic attached? I think the nice thing about SQL sometimes is that you can really, it's very easy to craft it to just what you want. Picture if I had done an entity command, and I would not wanna bring back the whole entity. So I'd have to done a select of just the properties I wanted there. And if I put that up in front of everybody and said, look how easy this is, and then showed the exact right lava, the perfect lava, I should say, they've been , yeah, right. That's super easy. Cool, John. It's sometimes easier just to show a basic example. Mhmm. Yep. That makes sense. Okay, Daniel. Last question. Are Rock agents global? Meaning agents we create can be used by anyone we give permission to. And if so, do they share context or is that per user? They are global. And I'm gonna qualify that for clarity. They're global to your Rock instance. They're not global to every other Rock instance in the world. So if you create an agent and give, , your staff workers role permission to that agent, all your staff workers can interact with that agent. They do not share context between people. We have the concept of sessions just you would see in Claude or other systems where you can switch sessions back and forth, and that switches your kinda your context history. But those sessions are unique to each person, so you don't have to worry about, , a session that I start, John accidentally picks up and starts using. Mhmm. Okay. Well, wow. We just went through, I mean, 20 questions or more. That was that was rapid fire. Yeah. John, is there anything you else anything else you wanna say to close this out? Yeah. As we're putting together the presentation for, this week, there is a lot of other topics that just had to get cut to the editing floor. We just didn't have time. So I just want to throw a few things. There's about a million things we could do. We could talk all day. The one thing I want to mention too is , we're not done with the skills. Just realize that is not a complete list and we understand that. We just try to give you the eighty twenty. And so we'll continue to add those skills and we just need feedback. So give us feedback as you're using it. We've already gotten feedback from churches. It's funny every time I watch a church who has access to it and they're doing demos and stuff when I'm watching, when they say, I'm gonna type this. And I'm , oh, no. , , you get worried as the person, , who's part of the project. And I've been I've been surprised at how well it's doing. Yeah. So I think our tool in SkillCall, had how many meetings do we have about that? I mean, dozens. Yeah. And then even the how we come up with a naming convention, it seems so easy when you look at it on the slide. None of that was easy. We were wrestling and learning and trying this. It's a long trial and error of, well, this naming didn't work, so let's try something else. Yeah. And one thing too I'd throw out there, when we started this, these models were dumb. , it was really hard to get into to not I mean, you give the same prompt and they would just do different things. And so we wrestled and wrestled and wrestled. And the naming back then was super important and the instructions were super important. Now, kind of important. So now the skills are there. I I wanna call out there actually is no skill to add a person yet. And we that's intentional because we just feel that could cause a lot of dupes. , let's roll let's slow our roll on that one. We do realize that's probably one of the next skills we need to make. But, that one just felt a little little dangerous. So that one will come. So, no, we didn't forget that. I also wanna mention too, , how we choose the models. So this again only relates to when you're using the chat within Rock. And we're using our Orchestrator and and then you go through the onboarding and then you pick standard or economy. And that's all you get to know. , not really gonna tell you what model behind the scene is because it's gonna be changing. Models are changing weekly. And we're not gonna be back there every day changing the model on you. But we're gonna be looking at the state of the industry. We're gonna be looking at token size, account context size, token costs, capability, speed. When we first started this a year ago, oh my gosh, it was kind of it was slow. It was very slow. Faster than we forget how many times we have to click to get to the person profile page, but we're doing stuff so it feels fast. Now it's much faster. But we'll be looking at speed. That's another thing we're looking at. There's all these factors that we're looking at privacy, 100. , we have to we're limiting ourselves to models that do not trace or track your data and do not train from it. So, it's private. So, that's how we're going to be choosing the models. Cost obviously is going to be a component. But that's again, I think a good thing because now you don't have to think about all that and do all that. I do think when it comes to privacy, it's something we have to be very careful about, but we are picking private models. But I do think sometimes we over I think privacy has become this mantra in AI that everybody says, well, you can't do stuff because it's not private. And in my opinion, it's , well, the contract says they're not training on it and they're not storing it and they're not doing anything of it. What is what is different than that than your email system or or your OneDrive or your Dropbox? , it comes down down to do you trust the contract you signed? If you don't, then don't sign the contract and don't use that vendor. But at a certain point, we already have that level of trust in all these other systems. To me, privacy as long as you've checked that box isn't really a concern. The bigger concern is, in my mind is the person behind the wheel. , is that person trustworthy? And what do I mean by that is , are they just outsourcing their work to it and just copy and pasting emails or are they actually using it as a helpmate, an assistant? That's the more dangerous thing to me. A lot of people are waving the privacy as a danger thing. I'd say, no. , check your contract. That's good. Okay. The bigger problem is the person driving it. , let's focus on that and train that person not to outsource their work. And we're gonna be talking a lot about that at the conference. So that's what that the AI and the Altar topic is all about. I think it's algorithm in the Altar. Yes. Perfect. Looking forward to that. Yeah. Mean, it's been a work in progress. I've been working on it for probably about three months. And I still have a small pack of what I'm talking about. And I have handwritten notes all over it. So every day I come back from my podcast, and I'm , okay, we gotta talk about this, this, this. Yeah. The last thing I want to talk about is a voice agent. Because we didn't really talk about how that works. That does require another service that the church will have to get. And we support two voice models or voice frameworks, should say. One is the OpenAI one and one is Grok. And they're priced differently. And price is a little bit tricky on those. They're not free. They're I wouldn't say they're expensive. If you look at the costs, I think you look at it go, you might think that is expensive, but you have to trade costs, convert costs to value. And I think if you think about the value of it's well worth it. But we're so used to just looking at the cost and we forget the value. One of the things that these the way they're they're build on is time. So OpenAI is, say you're driving in your car and you have it listening for an hour, you're not billed an hour. You only billed at what it heard and you talk to it. And so if you only talk to it for five minutes, billed for five minutes. And a lot of time in the conversation, nothing's happening. So even a five minute conversation, how much of that was actually doing stuff? There's a lot of pauses and you don't get billed for each little pause. So the costs are very hard to determine, I would say. So you're gonna have to experiment. Grok is the other way. Look at it , Oh, wow, this is super cheap compared to OpenAI. Well, they're going to actually bill you for the whole time. So, they're going to bill you for the hour. So, there's going be a lot of testing and research done for that. But we're not gonna provision that part simply because it's much, much more difficult to provision that. And it's not something we feel we can tackle right now. I think over time, they'll make it easier. So I just wanna put that pin in and call that foul. Got it. Well, I think those were really important points to walk through. So thank you. Anything else? Nope. It's been a lot of work. I wish people could understand how much effort it was. It's fun watching it change over time. I use fun in air quotes. Type two fun. There's sometimes it was fun, sometimes frustrating. Yeah. It's cool to see that it always got better, I think. But it's a very interesting time to be alive. Mhmm. Definitely. Alright. Well, thank you for joining us today. It was fun to have you. My pleasure. And, if you haven't yet, make sure you listen to the AI Summit. It's on our YouTube. Share it with your leadership and then come watch this episode so you can find all these golden nuggets. Thank you guys so much, and we'll see you next time.