Frequently Asked Questions

A few weeks ago a client messaged us at eleven at night, pretty rattled: he’d just gotten an alert from his AI provider saying usage had spiked in the last hour, something like two hundred dollars in forty minutes. Nobody at his company had touched anything. When we checked the project’s GitHub repo — public, because he’d pushed it himself without asking us, “to show a partner” — there it was, in a commit from three weeks earlier: the API key pasted straight into the code, no environment variables, nothing. Someone, or some scraping bot, had found it and was using it to generate who knows what on his dime.

An AI API key is basically a password that identifies your account to the model provider (OpenAI, Anthropic, Google, whoever) every time your app asks it for something. If that key ends up in someone else’s hands, they can use the model as if they were you, and the bill lands on you: the provider has no way of knowing it wasn’t you, unless you catch it fast enough to revoke the key.

What we did with this client was the usual playbook: revoke the key from the provider’s dashboard the moment we saw the alert, generate a new one, and this time put it in the server’s environment variables instead of code that gets pushed to a repo. That part took ten minutes. What took longer was convincing him that his partner didn’t need to see the actual source to understand what we’d built — a recorded demo would’ve done the job — and that a “private but with the key baked in” repo isn’t as private as it sounds, since any future collaborator, or a badly handled fork, inherits it just the same.

The bill, for what it’s worth, got refunded after we filed a fraud claim with the provider. That doesn’t always happen. It depends on the provider’s policy and how fast you report it, so it’s not something worth counting on.

Since then we’ve applied one rule on every project that we used to just assume everyone already knew, without actually checking: no key ever goes in the code, all of them live in an environment file that Git itself ignores, and production keys get stored separately, in the host’s secrets manager, never in a shared document or a WhatsApp thread about the project (we’ve seen that last one more than once). It’s the same lesson we took away from auditing our own server’s security: most scares don’t come from a sophisticated attack, they come from a key someone left lying around without noticing.

What I still haven’t figured out is how to explain this to a client without it sounding like a scolding, especially when the mistake came from someone with good intentions who just wanted to show off their project.

→ Direct link to this question

A client of ours who sells lamps and home decor online got in touch a few weeks ago. She wanted to automate part of the returns process: when a buyer sends a photo saying “it arrived broken,” she wanted the system to look at that image and decide whether to approve the return, instead of someone on her team opening every email and checking the picture by hand.

That’s exactly what multimodal AI does: it understands text and images (sometimes audio or video too) in the same conversation, without needing a separate system for each type of content. You send a photo along with a question, and it answers based on what it actually sees, the way you’d describe a picture to someone over the phone.

Until a couple of years ago, building something like this meant stitching pieces together: a vision model to classify the image, OCR if there was text in the photo, and then a separate language model to draft a response using whatever the other two handed it. Three systems, three places where something could break. With a multimodal model like Claude or GPT-4o, all of that collapses into a single call: the image sits in the same context window as the text, and the model reasons over both at once.

We had a working prototype for the client in an afternoon, literally. We fed it the photo plus a prompt with her return policy’s conditions, and it comes back with whether there’s visible damage, what kind, and a short note for the team. It handles obvious breakage well (a shattered piece of glass is easy to catch) and struggles more with subtler stuff: a small scratch on a matte finish gets flagged some of the time and missed the rest. We still keep a human checking before anything gets approved automatically. The AI cuts down the work of eyeballing every photo one by one. It doesn’t replace the final call.

What almost nobody asks upfront, and should, is what happens to those photos once they leave your server. They go to a third-party provider’s API, same as with any language model, so if a customer uploads something with visible personal details (an address on a label, a face in the background) it needs the same care as any other data you hand over to an AI. In this case we told the client upfront that return photos go through that process, and put it in writing in her policy, not as a footnote.

We haven’t tried feeding audio into the same flow yet, though the same client has already asked whether something similar could work for the voice notes people sometimes send explaining what happened. Technically, sure, it could. Whether it’s worth building for the volume of returns she gets is a question I don’t have a clean answer to.

→ Direct link to this question

A client wrote to me a few weeks ago, pretty annoyed: he’d asked ChatGPT to summarize the same document twice, with the exact same prompt, copied and pasted, and got two different summaries back. He thought the tool was broken. It isn’t. It works like that by design, and it took me a while to accept that too, the first time we built an integration with a model for a client and the results in testing just wouldn’t come out the same twice.

A normal program is deterministic: same input, same output, every time. A spreadsheet doesn’t have good days and bad days. A language model doesn’t work that way underneath. When it generates text, at every single step it computes a list of candidate words with a probability attached to each one, then picks one from among the most likely, not always the single most likely word, but one sampled at random from within that range. That’s controlled by a setting almost every API exposes called “temperature”: higher means more variety (and more risk of wandering off); lower, or set to zero on some models, means answers much closer to each other, though almost never a hundred percent identical. In one line: an AI gives different answers to the same question because it doesn’t always pick the single most likely word — it samples among several candidates based on that randomness setting, so some variation isn’t a bug, it’s part of the design.

Why build it this way, when it makes the tool feel unreliable? Because the same mechanism that makes two summaries come out different is what lets the model avoid sounding the same every time, lets it finish a sentence naturally instead of repeating the statistically flattest word combination over and over. Without that randomness a chatbot would sound like a machine, not like something imitating a real conversation.

The trouble shows up when someone, us included the first time, uses a model for a task that actually needs to be deterministic. We built an automation for a client that read invoices out of PDFs and pulled the amount, date and supplier using an LLM. It worked fine in testing. In production, every so often the exact same invoice format would come back with a field misread that had come out perfect the time before. It wasn’t a bug in our code, it was the model being a model. We dropped the temperature to the minimum and added a validation layer with fixed rules, the amount has to be a valid number, the date has to actually exist, and the problem got a lot better. It didn’t disappear entirely. With these tools, “almost always right” is not the same as “always right”, and you need to design knowing which one you actually need for each piece of the process.

For normal use, asking something, getting ideas, having it help draft an email, this variability doesn’t matter at all, it even helps. The problem is when someone builds an automated business process assuming an LLM behaves like a spreadsheet formula. It doesn’t. The sooner you have that straight while designing the automation, the fewer surprises with weird data you’ll run into six months after switching it on and walking away.

→ Direct link to this question

Last autumn a real estate client asked us for something that sounded simple: an assistant that could answer questions about their earnest-money contracts, just by uploading the PDF straight into the chat. We tried it first on a short contract, eight pages, and it worked great — questions about deadlines, the penalty clause, who keeps the deposit if the deal falls through. All fine.

Then we tried one of their genuinely long contracts, the kind dragging technical annexes and a whole history of amendments: forty-odd pages. The model started answering with total confidence about clauses that didn’t exist. Not slightly wrong — it was inventing entire article numbers.

That’s where the context window comes in, a concept almost no client has heard of until it blows up in their face. Every AI model has a limit on how much text it can “hold in mind” at once, and that limit is measured in tokens, not pages or words (a token is roughly three or four characters). A model with a 128,000-token window can handle, roughly, some 300 pages of normal text. Sounds generous. But a scanned PDF, with badly converted tables or embedded images, can end up costing two or three times the tokens it looks like it should, and that’s exactly where the margin quietly runs out.

And here’s the part that actually matters: when you go over the window, the model doesn’t throw a clean “this doesn’t fit” error. Depending on the tool, it truncates the document wherever it can, or silently summarizes chunks without telling you, or just fills the gaps with whatever seems statistically likely given the pattern of the text. That’s what happened to our client: the model never had the end of the contract in its active window, and instead of saying “I don’t know,” it improvised.

The fix wasn’t to grab a model with a bigger window (they exist, some with a million tokens, but they cost more and run slower). We went with RAG instead: rather than dumping the whole contract in one go, the system first pulls the fragments relevant to the actual question, and only those go into the context window. Less elegant than “upload the PDF and ask anything,” but a lot more reliable, and cheaper too, since you only pay for the tokens each query actually uses.

If someone tries to sell you a “no limits” AI assistant for long documents, ask exactly what happens when a document doesn’t fit whole. If the answer is vague, they probably haven’t tested it against a real forty-page case yet.

→ Direct link to this question

A few months back a client in the legal sector asked us for something very specific: they wanted an assistant to help their team search through thousands of pages of case files, but they flat-out refused to let any of those documents leave their server. Not to OpenAI, not to Anthropic, not to anyone. That’s exactly where the difference between “cloud” AI and self-hosted open-source AI comes in.

When you talk to ChatGPT or Claude, your questions travel to a server run by OpenAI or Anthropic, get processed there, and the answer comes back to you. Convenient, fast, and you never have to think about infrastructure. The catch is that for some businesses, sending sensitive information to a third party simply isn’t an option — whether because of a contract, industry regulation, or the client just not trusting it. In one line: running AI in the cloud means sending your data to a third-party server (OpenAI, Anthropic, Google) in exchange for convenience; running it on your own servers means that data never leaves your network, in exchange for building and paying for the infrastructure yourself.

That’s where open-source (or “open-weight”, the more precise term) models come in: Meta’s Llama, Mistral, Qwen. Models you can download and run on your own hardware, so not a single word of what you ask ever leaves your network. With tools like Ollama or vLLM, setting one up isn’t nearly as complicated as it sounds — a couple of days of work, not weeks.

What does cost real money is the hardware. For the law firm’s project we ended up renting a GPU machine from a cloud provider (buying your own graphics cards is something we almost always advise against), and even then the monthly bill ran about four times what using the Anthropic API would have cost for the same volume of queries. And honestly, the quality of the answers was worse: Llama does a decent job, but it’s not in the same league as Claude or GPT-4 on complex reasoning over long text.

So the real question isn’t “open source, yes or no”. It’s “which matters more to me, confidentiality or quality”. For most of our clients the answer is clear: using a serious provider’s API, with a solid data processing agreement in place, ends up cheaper, faster to set up, and gives better results. The law firm was one of the rare cases with no way around it: their own lawyers had confidentiality obligations that ruled out any middleman, however good the contract on paper.

There’s a middle ground people don’t talk about as much: some providers (Azure OpenAI, Amazon Bedrock) offer the same closed models but with contractual guarantees that your data isn’t used for training and isn’t shared with anyone else. For a good number of clients worried about privacy, that solves the problem without standing up your own servers or giving up quality. We’ve yet to explain this to someone without watching their shoulders drop a little.

What I honestly don’t know is what happens in two years, once open models close the quality gap with the closed ones. Maybe “self-hosted or not” stops being a question of sacrificing quality and turns into a purely operational cost calculation. Or maybe the big models just stay one step ahead forever. We’ll see.

→ Direct link to this question

A client who resells secondhand industrial machinery asked us a couple of months ago if we could “train an AI” on fifteen years of emails and quotes sitting on a server. He wanted an assistant that could answer his sales reps’ technical questions using that history. I told him something like that was doable, but “train” was the wrong word, and that if anyone offers to “train your own model” for two thousand euros, he should be suspicious.

Training a model from scratch, or even fine-tuning an existing one, takes a lot more data than a typical small business has lying around, and the result usually isn’t any better than the cheaper option: giving the model access to your documents at the moment someone asks a question, instead of trying to get it to memorize them. That’s called RAG (retrieval-augmented generation), and it’s what almost any agency actually builds when they talk about “AI with your company’s data,” even if they don’t always call it that. As a bonus, that way the model answers with what your actual document says instead of trying to memorize it and risk making things up.

The practical difference is this: with fine-tuning you’re modifying the model’s weights, which means GPU time, training runs, and having to retrain every time your data changes. With RAG you store your documents, chunked up, in a vector database, and when someone asks something, the system pulls the most relevant chunks and hands them to the model as context before it answers. Adding a new quote is literally just uploading a file. No retraining, no GPU bill.

For the machinery client we built exactly that: their emails and quotes chunked and dropped into a vector database (we used pgvector since they already ran Postgres, no need to bolt on a new tool), plus an internal assistant a rep can ask “what did we quote client X for that forklift model back in 2024?” and get the relevant document back with the answer already summarized. It cost a fraction of what fine-tuning would have, and we had it in production in two weeks.

There is one case where fine-tuning actually makes sense: when you need the model to always answer in a very specific format or tone and RAG can’t force that on its own (say, generating code in a particular internal style based on thousands of existing examples). But that’s rare in the kind of work we do for small agency clients, and when it has come up it’s almost always been for a very specific content-generation case, not “answer questions about my data.”

What bugs me most about this whole topic is how many proposals I’ve seen floating around offering to “train your own custom AI” when what’s actually being built is a RAG setup with a handful of lines of code wrapped around it. It’s not exactly a lie, but it inflates the price and makes people think they need far more infrastructure than they actually do. If someone pitches it that way, just ask directly: is this fine-tuning or RAG? The answer tells you pretty quickly whether the person selling it to you knows what they’re doing.

→ Direct link to this question

A client emailed us a couple of weeks ago asking for “a chatbot to handle orders.” Ten minutes into the call it was clear that wasn’t what they actually wanted. They wanted something that checked stock, worked out shipping, and, if needed, pinged the supplier. That’s not a chatbot anymore. That’s an agent.

The short version: a chatbot replies. It gets a message, whichever model is running generates an answer, and the loop ends there. An agent can decide to do something before it answers: query a database, call an API, run a script, and only then give you a response that actually accounts for what it found. The key isn’t the model itself, it’s that the model has access to tools and can choose to use them.

This is called “tool calling” or “function calling” (the idea behind protocols like MCP), and it’s been around for a while. What’s changed in the last few months is that it’s become reasonably reliable. It used to make up parameters or call the same API three times for no reason. With Claude Opus or Sonnet, or GPT-4.1 onward, it fails a lot less. Not never. It still happens, especially with poorly documented tools or ambiguous parameter names.

For this client we built something fairly simple: an agent with three tools. Check stock, calculate shipping cost by weight and postcode, and draft a supplier alert when stock dropped below a threshold. No free-form language in the critical part: the agent decides which tool to call, but the shipping calculation is plain code, not something the model “reasons” its way through. Watch out for this one, because a lot of people skip it on their first agent build: the model decides the what, the code handles the how. Let the model guess shipping rates and you’ll get surprises —annoyingly hard to debug ones too, since the model won’t explain why it got it wrong, it just returns a different number each time.

The hard part wasn’t the AI. It was deciding what happened when the agent picked the wrong tool or got stuck halfway, say on a timeout from the supplier’s API. We had to add retries and, more than anything, a step limit so the agent wouldn’t loop calling the same tool twenty times if something broke. The first version we tested didn’t have that limit. In an internal test it sat there querying stock in a loop for almost two minutes before we killed it by hand.

If you’re weighing something like this for your business, the question that actually matters isn’t whether AI can do it (it almost always can, at least in a demo). It’s what happens the day the tool it calls fails, returns something odd, or takes fifteen seconds to respond. That part isn’t solved by the model. You solve it.

→ Direct link to this question

The first time a client asked me “wait, why are you charging me per word?” I had to stop the meeting and explain that it’s not words. It’s tokens, and the difference matters more than it sounds like once the API bill lands at the end of the month.

A token is a chunk of text, almost never a whole word. “Cat” is usually one token. “Extraordinarily” might split into three or four. Models from OpenAI, Claude or Gemini don’t read letters or loose words: they read these fragments, and every API call gets billed on how many go in (the prompt you send) and how many come out (what the model replies). Spanish, as it happens, tends to burn more tokens than English to say the same thing — we found that out the hard way on a hospitality client’s project, when the monthly API spend came in almost 40% over what we’d estimated using English test copy. In one sentence: a token is the smallest unit of text — usually a word fragment, not a full word — that an AI model uses to read and generate language, and it’s the exact unit each API call gets billed on.

That has consequences that shape how you design an AI feature, not just what it costs:

• A long system prompt (the fixed instructions you send the model on every request) gets billed on every single call, even if the user only typed two words.
• If a chatbot keeps the full conversation history to fake “memory,” every new message drags the whole thread along with it, so the cost per turn climbs as the chat goes on.
• The more capable models don’t just charge more per token — left unconstrained, they also tend to write longer replies, so the cost stacks twice over.

When we budget an AI integration for a client, the first question isn’t “how much will AI cost” in the abstract anymore. It’s estimating tokens per interaction. We take real examples of what the user is going to type, run them through each provider’s tokenizer (they all publish one, free to use) and multiply by the expected monthly volume. It’s a tedious step. It’s also the one that stops surprises.

There’s a detail almost nobody asks about that more people should: a model’s context limit is also measured in tokens, not “messages” or “pages.” If a client wants the AI to “read the whole product catalog” before answering, that catalog has to fit, in tokens, inside that window. If it doesn’t, you either chunk it up or build a vector store that pulls in only what’s relevant per query — and at that point it’s a different project with a different budget, not a box you tick in the first meeting.

What still bugs me is that almost no provider shows you a live token count while you’re typing the prompt, as if they’d rather you not think about it until the charge hits the card.

→ Direct link to this question

A logistics client wrote to us a couple of months ago, pretty on edge: he’d read that a major AI provider had hiked prices for certain enterprise customers with no warning, and wanted to know if the same thing could happen to the assistant we’d built him for sorting delivery incidents. I told him the truth: yes, it could. We hadn’t talked about that risk when we signed the project. Neither had he. Neither had we.

Bringing an AI model into a business means taking on a new dependency, and a serious one. It’s not like depending on hosting, which you can swap out in an afternoon by copying files from one place to another. Once an assistant has been running for months on a specific model, the instructions you give it (the “prompt”) are tuned to that particular model’s quirks: how it reads dates, how much it rambles if you don’t set a limit, what examples it needs to stay on script. Switching providers isn’t swapping an API key. It’s re-testing half the assistant’s logic from scratch. In short: if your business depends on a specific AI provider and that provider raises its price, switches models, or drops support, migrating isn’t just swapping an API key — it’s reworking the prompt and re-testing a good chunk of the assistant’s logic.

It happened to us for real with an insurance client. The model we were using got deprecated in its old version, and we had three weeks to migrate the assistant to the new one, on a deadline set by the provider, not by us. The general logic still worked, but the tone changed: the new model was drier, gave shorter answers, and we had to rework the whole prompt to get it sounding like it used to. Three days of work nobody had budgeted for.

Since then I try to explain this on every new project that leans on AI in a serious way, not as a shop-window feature. Two things help a lot. One: don’t put business logic inside the prompt if you can avoid it. Hard rules (what price applies, what deadline has to be met) belong in ordinary code, and the model only handles the language part. That way, if the model changes tomorrow, what breaks is the tone, not the calculation. Two: write the code so that switching providers means swapping out one function, not rewriting the whole application. Nothing fancy, but few clients ask for it, because nobody’s told them they need to.

What I’m still not sure about is how much of this is worth building in from day one on a small project. That abstraction layer costs hours the client sees on the invoice, and for a simple assistant at a small business it might be paying for insurance you’ll never need to use. For the insurance client, looking back, it was worth it. For the delivery-incidents one, honestly, I still don’t know.

→ Direct link to this question

A couple of months back a client who sells clothes online asked if we could generate all forty product shots for the next season with AI instead of booking the photographer again. It would have saved him something like €3,000 and two weeks of waiting. I told him yes, technically it’s doable, but “legal” and “no problems” aren’t the same question, and answering it properly took more digging than I expected.

The copyright part turns out to be simpler than people think: in Spain and the EU, something generated entirely by AI without human creative input doesn’t get copyright protection, nobody counts as the “author” in the legal sense. That doesn’t mean it’s automatically public domain or free to use however you like, it comes down to the tool’s terms of service. Midjourney, for instance, hands commercial ownership to the paying user; other tools keep it, or share it. Before we drop a single generated image onto a client’s site, I check that specific tool’s terms. I don’t assume they all work the same way.

But what actually worries me isn’t the intellectual property side. It’s resemblance. I explained to the client that an image generator can spit out something that looks too close to an existing stock photo, to a garment with a registered pattern, or, worse, to a real recognizable person, and that’s where you risk infringement or image rights, no matter what the tool’s fine print claims about ownership. Text has the same problem from a different angle: ask a model to write up a product’s technical specs and it invents a certification or a feature the product doesn’t actually have, and now the legal issue isn’t intellectual property at all, it’s misleading advertising, and that lands squarely on the client.

We ended up settling on a middle ground: AI generates the background and composition of the product shots, but the garment itself stays a real photo, composited in, and any generated text goes through a human review pass before it’s published — an approach that also helps keep the variable monthly cost of running AI in check, not just the legal risk. No inventing features, full stop, because saving on a photographer isn’t worth walking into a lawsuit. We use AI for this blog’s own header images, the code running this site generates them automatically whenever I publish a new post, and I don’t lose sleep over that one because they’re abstract illustrations, not product shots or people, nobody mistakes them for something real.

There’s another angle almost nobody brings up: if a client asks you to never mention anywhere that the content is AI-generated, and someone finds out anyway, the reputational damage usually outweighs whatever the underlying legal risk was — not unlike what happens when nobody stops to read what actually happens to the data you hand an AI: the fine print matters more than it seems. I’m not a lawyer and this isn’t a closed legal verdict, it depends on the tool, the country, and the specific use, but it works as a rule of thumb: the closer the generated thing gets to something or someone identifiable, the closer you are to a problem, legal or just a credibility one.

→ Direct link to this question