Logo thisisvoid.in
Created: 2026-11-11 Updated: 2026-11-11 37 min read

Sevi: shipping a personal tool for $0

I’ve had this problem for a while now. Back in India, I was a blog person over a video person and I had enough time to sit and read through all the blogs and keep myself updated. At the same time, I walk a lot, and I consume a lot of podcasts while walking, because I don’t enjoy listening to music with headphones on (I’m more of a speaker music enjoyer guy). Now life is so hectic that I have absolutely no time to catch up on anything. It takes a solid 25-30 min to get through a blog, and sadly I can’t afford 25 min of undivided attention. The blogs I consume aren’t just purely technical ones, which I’d take my time to focus on and read. I also consume a lot of non technical stuff, product stuff, and technical but not deeply technical stuff like hot takes. These blogs don’t need my full focused time. At the same time, since NYC is extremely walkable, I spend almost 45 min a day just walking around. So I wanted to build a tool for myself to solve this problem.

demo link

solution to my problem

To immediately solve my problem and get started, I want a tool that can:

These are my initial problems that I need solved asap, and it has to be built in such a way that it’s easy to add features on top of it whenever I want one.

The problem statement is very simple and straightforward: 4 fixed and clear features to build, and more importantly, I can take the liberty to make features and tradeoffs specific to me. Building generic software is always more difficult than building very specific software. By specific software I mean I exactly know the usage pattern, how I will use it, and the fixed semantics and rules I’ll use the software with. For example, if I reply to the source blog, that will be treated as a follow up question.

So I want to take the deployment a step further. Since I’m the only one who’s going to use it, and I know exactly the access patterns, how I’ll use it and what kind of workload it is, I want to challenge myself and see if it’s possible to set up a proper secure infra that at the same time doesn’t cost anything. I’ll anyway spend money on a good text to audio model for our blogs, so that’s enough motivation for me to go do a trade study and stitch together various pieces that have generous free tiers, and design it in such a way that it never exceeds those free tier limits for our workload.

So the goal is simple: secure and free infra

With our goal set, we can go backwards and try to come up with primitives and a design that have acceptable tradeoffs but help us get to our goal of no cost infra. This blog is going to be about all the choices and tradeoffs I made for such a simple use case.

decision 0: the non negotiables

decision 1: no EC2

The easiest way to deploy our code is to boot a VM and run our code in there. Whenever we use our user interface, it sends a request to our server. The code running in our VM makes the LLM API calls, stitches the audio together, stores whatever state needs to be stored in the DB, and then sends out the audio. ec2 This is super easy.

The problem with this setup is that a VM costs money. Most providers either don’t have free tier VMs, or the ones they have are unreliable and shady. One common pattern with virtual machines is that they give you a VM for a particular period of time, like a one or two month trial, and then start charging you for the instance. That goes against our idea of absolutely no cost, because for a month or two we might not pay anything, but then we need to start paying for the instance. Also, if you look at the components of a virtual machine, there’s a minimum disk size that gets allocated, and that’s more than we even need. For example, 20 GB of disk space is of no use to us, because an audio file is 4-5 MB, and why would we need to retain audio for so long anyway?

Having a fixed machine is easier to build for, because since the machine is fixed, a lot of things around it can also be fixed. For example, for this specific use case, you can just use an embedded database like SQLite inside the machine, and I’d be okay with that. If I want a backup, I could always do SQLite WAL replication using Litestream and keep the backup somewhere. That’s also fine. Maybe since none of these audios are very important (they’re just blogs, we can always regenerate them), we don’t even need to back up the database. Even if we want to store the audio, we could store the files in the 20 GB or whatever volume the EC2 / virtual machine has by default, so whenever I ask for the same blog again, we just take it from there and give it back. Stuff like that are some advantages of using a fixed machine, because it solves a lot of problems. The only downside is I need to pay some money for a workload that I’m not even using most of the time. There will be days at a stretch where I never read a blog, and I’ll be paying for something I don’t even use.

decision 2: serverless

Another reason why a fixed VM is a bad idea for my use case is resource utilization. At max in a day, I would generate 1, 2, or maximum 3 blogs. If you think about it, each blog takes a while to listen to, so at max I could get through like 3 blogs a day, correct? So why run a machine 24⁄7 when you only need it for a few minutes a day? Which is super stupid, right? So say hello to serverless.

As per the definition, serverless means somebody else is managing your infrastructure and servers, but that’s not the point here. The reason people choose serverless is when they have bursts of workload, and serverless is billed for the amount of time you run that task. So if I’m running like thrice a day, for maybe 2 minutes each run, I’ll effectively be running about 6 minutes a day, which is nothing. You pay for the kind of machine your serverless boots and for the amount of time the code is running, and since you only run for such a small time, it’s efficient for this use case.

Most serverless providers have a free tier but at the same time the odds of you misconfiguring it and misdesigning your software are really high. Also, at scale, serverless eventually gets more costly for such usecases. So the moment your application gets remotely famous, the odds of you paying an insane amount of money are really high. They have fixed limits on number of requests, compute time, memory time, and outbound data transfer, and if you want to stay in the free tier, you have to make sure you don’t cross any of them. That’s the difficulty here. But if you have very specific use cases like this one, serverless can be really useful. With a virtual machine, you prepay for the instance’s throughput and specs, so at the end of the month you pay a fixed amount. With serverless, the cost depends on the load that goes into it, so it varies with the load. So one stupid misconfiguration or leak and boom, you pay 10 grand.

I went ahead and looked for serverless options with generous free tiers, because I was very sure this is a nothing workload, and if designed properly, we don’t have to pay for anything. The available options are AWS Fargate, AWS Lambda, Cloudflare Workers, and Google Cloud Run. Google has a whole suite of serverless products, and for our use case Cloud Run is the one that fits (we’ll see why in decision 4). I finally decided to go ahead with Google Cloud Run, because it provides a monthly Always Free tier for request-based billing that resets every month:

For my use case, I would never use even 50,000 requests, because I’d make 5 to 10 requests per day. And 180,000 vCPU-seconds is roughly 50 hours. There will definitely not be 50 hours of CPU time. I first had a local version of it, and on testing I could see that a 20-25 minute long blog (which is rare, because blogs are usually a lot shorter) would take roughly 2-3 minutes to process.

The outbound data transfer might look a little problematic, but since it’s just an audio file of a few MB, I don’t think I’ll exceed 1 GB. The free 1 GB counts traffic leaving a North America region, so as long as the service runs in a US region, it doesn’t really matter where the receiving end sits. I’ve set up Telegram as my user interface, and the audio files are around 5-6 MB after all the post-processing, so each time I send one back to my Telegram adapter, it’s only a few MBs. So I’m very sure I won’t cross 1 GB per month.

If you ask a more seasoned software engineer, the first thing they’d say is that serverless plus some persisted state is a bad design. Indeed, it is a bad design, because there are a lot of moving parts here that you need to meticulously build to get things right. I reasoned out why I chose it anyway, but you can see that things would have been a lot easier with a virtual machine, and because of serverless I’ll be adding a lot of parts just to make it work, which I’ll explain below in this blog. It would have been a lot easier with an EC2 instance and a persistent database. Serverless is also stateless: they assign you some machine, and once the task is done, the machine eventually dies. So when you want to add some kind of persistence to a non-persistent system, all the techniques you apply to it are going to be a little bit hacky. But serverless plus a persistent database is a common pattern these days, because people have figured out how to set things up, and most AI workloads are really inconsistent and bursty, so people are using this pattern a lot, mainly for cost reasons. Since this is free for me, I thought I could go build one with the same pattern. Because I chose this, I had to add a lot of components around it to make sure the serverless part works, and code-wise and design-wise there had to be a lot of changes to accommodate serverless, which we’ll cover below.

decision 3: telegram

Now that we’ve decided we’re gonna run on serverless, let’s shift the focus to the user interface first: how the user interface is built and how the APIs are built. Then we can go completely into the design. This is a very personal product that I’d use every day, and when something is very personal and you use it every day, it has to be on your phone. That’s the most obvious one. The second one is that it has to be extremely accessible. If you look at all the personalized agents that have been built in 2026, most of the ones that worked are either integrated with WhatsApp (or whatever messaging app you have) or have their own app. I did some thinking about it, because Hermes, Instinct, and all of them didn’t work because of some out of the box thinking. They worked because they’re way too accessible: you just open WhatsApp, open Instinct, ask your question, and get the answer. So I want that kind of interface for me to use Sevi.

The problem with such interfaces is that you’re limited in what you can do, because a chat interface can only offer certain things. You can send some commands, reply to a chat, maybe delete a chat, or create a fresh chat, stuff like that. These are very limited things. So what if I need some other feature that a chat interface can’t offer today? What I thought is, I need to separate this interface layer from the Sevi core itself, and the interface belongs to whoever wants to build on top of Sevi.

So my Sevi core exposes certain APIs, and those APIs can be accessed by any sort of interface. If you put Telegram or Discord in front of Sevi, the Telegram bot or the Discord bot should be able to reach the Sevi core through that API and get the response back through the same API. I foresee myself building a small web app for this, because I’ll be able to track a lot of stuff with a web app that you can’t with a chat-like interface. So the idea from day zero is that the Sevi core is separate from the interface, and every kind of interface uses the same API endpoints. That means the API endpoints have to be designed generically, so that anybody can access them.

In 2026, if you want to build some personalized application for yourself, Telegram is the way to go according to me, because it has a lot of primitives you don’t need to vibe code or build again. It has a beautiful chat interface, and it supports webhooks. Assume you have a web app, and you’ve generated the audio: how are you gonna send this audio to your web app? You can’t just transfer an audio file directly from your machine to the web app, right? But with Telegram, there’s an endpoint where you send the audio file, they store it on their side and stream it back in the Telegram interface, and we don’t have to worry about any of it. Stuff like this are the advantages of using well established user interfaces like Telegram or Discord. It’s easy for me to use every single day, because all I need to do is open Telegram, go to my bot, paste my URL, and things work, rather than worrying about another UI I need to maintain. I’ll eventually build one, but for now Telegram feels like the right way to build, and if you’re building something personalized as well, you should also use a Telegram bot as a starting point.

decision 4: container image

To see why we need a container image here, let’s look at the parts in this code. First, this is a Golang application, so we have a binary that needs to run. It’s not just a code file you run in a Lambda, or a TypeScript file you run on a Worker. This is a proper Go program compiled into a binary. The next component is ffmpeg. The audio is chunked and fetched from the text to speech model, and the chunks need to be assembled, cleaned up, downsized, and compressed if needed, plus all the other audio-related processing. So I need ffmpeg here.

On a traditional VM (such as an EC2 instance), setup happens once. You clone the code onto the machine, install ffmpeg and the other runtime dependencies, and start the application. The process keeps running, so every later request goes to an environment that’s already set up.

Serverless platforms work differently. Instances are created when traffic arrives and removed when it stops, so any new instance may start from a blank environment. If that environment had to clone the repository from GitHub, install ffmpeg and the other dependencies, and then start the application, all of that would happen during the cold start, which is the time between a request arriving and a fresh instance being ready to serve it. Cold starts add latency for users, and on most platforms you’re also billed for startup time. Since this is a user-facing tool, both matter.

To avoid this, I moved all setup to build time. A multi-stage Dockerfile compiles the Go code in a builder stage. The final image contains only the compiled binary and the runtime dependencies, such as ffmpeg. The Go toolchain isn’t needed at runtime, because Go produces a self-contained binary. I push this image to a container registry (Artifact Registry on Google Cloud). Each new serverless instance then starts from the prebuilt image with everything already installed, so the only startup work is launching the binary. Instances that are already warm skip even that and handle requests immediately. Also, if you want to scale out in the future, Cloud Run prepares and caches the image at deploy time. When traffic rises, new instances start from that cached copy instead of pulling from your registry each time. Keeping the image small (slim base image, multi-stage build) still helps cold starts.

It’s also important to keep the multi-stage Docker image extremely lean, because Google’s Docker registry, Artifact Registry, has a free tier limit: you can store up to 0.5 GB for free. So I made sure the image is extremely small and contains only what’s needed, so it never exceeds the limit. Also, when there’s an upgrade, I don’t keep multiple versions of the image. I store only one image at a time. In a company, or if you’re building a real production system, you should keep multiple backup images, with the latest one tagged as latest, so that whenever there’s a problem with the image you can swap it. But since I don’t have that problem (I can always rebuild the image and push it again), I didn’t do that, to save space and cost.

So the flow until now is: user interface, goes to serverless, our serverless (Cloud Run) uses the Docker image and runs the executable in the image.

decision 5: secrets

I was deciding which secret manager to use here. I keep saying this: even if folks use an EC2 instance, they should definitely use a secret manager. But very surprisingly, I see many production systems that have an EC2 instance take it as leverage and start using .env files, which is so stupid. What if you want to rotate the key? What if you’re in a compromised position where you can’t SSH into the machine, but you want to change the .env? Okay, that can be a hypothetical situation, but multiple folks SSHing into the same machine, so that all of them have access to that secret env, is a very common scenario, and I’ve seen this misused in the past. Especially with LLM credentials now, you have to be even more careful securing secrets, else the odds of getting compromised are even higher than before, because everybody wants LLM API access now. A secret manager is very, very cheap, so there’s no reason not to use one.

Google Cloud Secret Manager includes a permanent monthly free tier allowance per billing account:

Any app will have a lot of secrets, right? For example, the API keys, here the Telegram secrets, some security-related secrets, and the database connection secrets. These would definitely cross six secrets. But I remembered one trick an old school guy who runs an insanely huge SaaS company in India told me. This trick is now really common: you store all of your secrets as one single JSON, so there’s one key, and the value is the whole JSON of secrets. You fetch that in your system and unmarshal it. He also told me a funny story about the limit on how much data you can store in a single secret (I think for Google it’s 64 KiB). They were on AWS and hit AWS’s single secret size limit, so what they did was keep compressing the secret (zstd) to free up more and more space, and then stuff more data into that key. So I just reused that idea. Instead of storing each value separately in the secret manager, I have only one secret, a huge JSON, stuffed into the secret manager, and I unmarshal it locally. That’s how I dealt with secrets. So the secret manager is also free.

decision 6: database

So why do we need a database for Sevi? Just for useful stuff, like storing the audio link (if there’s a web app tomorrow, we need the audio link), and we need to track the progress of which blogs have been converted, the usual stuff.

Because we have a serverless machine and some job is running on it, the state of that job is also very important. If it abruptly stops, we have to resume from wherever we stopped, and problems like these need to be solved. Initially, my idea was to build just a local version, and I kept using it for a couple of days to see whether this whole Sevi idea was workable for me: whether I’m actually using it, whether I like the voice and texture, and whether I like this flow of listening to blogs and asking them questions. My setup was very straightforward, and I don’t feel ashamed about it, because my favorite database is SQLite. If I’m building anything for myself, I don’t carry any headache, I just build it with SQLite. And when there’s an EC2 instance, a fixed VM, that’s easy, because you can have the embedded database on the VM and you’re good to go. But the moment things become serverless, this becomes a little tricky.

An embedded database is supposed to be used as an embedded database. Since we now have serverless here, we’re decoupling the machine from the database, so SQLite might not be the best choice. But I tried my level best to figure out whether we could still use SQLite. The first thing I did was design a cheap alternative myself, which is kind of a bad idea. Here’s how I designed it: you have a serverless machine, you enable write-ahead logging in your database, and every 2 seconds you back up the WAL to your object store. So there’s always a copy of your database somewhere else, and you can recreate it whenever you want. The flow would be: a serverless instance boots up from the Docker image, pulls the backup from the object store, recreates the database, and then uses it. It’s all time consuming. Also, if you back up the WAL to your object store every 2 seconds, what happens if there’s a network connectivity issue? You just lose the last backup, and stuff like that. It also becomes even more complex to manage all the pieces, because we don’t know how long it takes to fetch the data from the object store. Since we keep writing to the object store, we need to pull from it, which takes some time, and then we have to recreate the database. All of this counts as part of the serverless runtime, so the runtime goes up too. This idea works when you have an EC2 instance with single tenancy, where every VM has its own embedded database and you back it up, and there it will definitely work well. But for this use case it’s a bad idea, so I ditched it immediately the moment I drew the design.

wal

So then I looked into hosted SQLite-compatible solutions. The two famous ones out there are Turso (which Supabase recently announced it’s acquiring) and Cloudflare’s D1. Both offer a generous free tier, but I had problems with both. First, Cloudflare D1 doesn’t let you run transactions the normal way (BEGIN … COMMIT), which is extremely problematic here. Take a simple Sevi example: when a blog is done, I insert the audio row, take its new id, and mark all the chunks as stitched with that id. Either both happen or neither does. On D1, this fails on the very first line:

await db.exec("BEGIN TRANSACTION"); // D1_ERROR: Cannot use BEGIN TRANSACTION
const { id } = await db.prepare(
  "INSERT INTO audio (blog_id, r2_key) VALUES (?, ?) RETURNING id"
).bind(blogId, key).first();
await db.prepare(
  "UPDATE chunks SET audio_id = ?, status = 'stitched' WHERE blog_id = ?"
).bind(id, blogId).run();
await db.exec("COMMIT");

D1 rejects explicit transactions, because one open transaction could block the whole database for everyone else. Instead, they want you to send all the statements together as one batch, and the batch runs atomically. The catch is that your code can’t read anything in the middle of a batch, so I can’t get the id back and use it in the next statement. I have to rewrite the query so it finds the id by itself:

await db.batch([
  db.prepare("INSERT INTO audio (blog_id, r2_key) VALUES (?, ?)")
    .bind(blogId, key),
  db.prepare(`UPDATE chunks
              SET audio_id = (SELECT id FROM audio WHERE blog_id = ?),
                  status = 'stitched'
              WHERE blog_id = ?`)
    .bind(blogId, blogId),
]);

In Postgres, none of this drama exists, and the reason is how the two are built. Postgres is a client-server database your app holds a connection, and the transaction state lives on the server for that connection, so the transaction can stay open across multiple round trips while your code does stuff in between. And because of MVCC, an open transaction doesn’t freeze everyone else: readers still see the last committed data, and other writers are only blocked if they touch the same rows. D1 is SQLite underneath, and SQLite allows only one writer for the whole database at a time. Each D1 query is a separate call from your Worker, so if D1 let you keep a transaction open across those calls, every other write to the database would be stuck waiting until you commit. That’s why they force everything into a single batch that runs and finishes in one go.

So in Postgres, you open a normal transaction, read the RETURNING id, run whatever Go logic you want in between, write, and commit. If anything fails, roll back and everything is undone:

tx, err := pool.Begin(ctx)
if err != nil { return err }
defer tx.Rollback(ctx) // no-op if already committed

var id int64
err = tx.QueryRow(ctx,
  "INSERT INTO audio (blog_id, r2_key) VALUES ($1, $2) RETURNING id",
  blogID, key).Scan(&id)
if err != nil { return err }

if _, err = tx.Exec(ctx,
  "UPDATE chunks SET audio_id = $1, status = 'stitched' WHERE blog_id = $2",
  id, blogID); err != nil { return err }

return tx.Commit(ctx)

So with D1, every read, decide in code, then write flow has to be bent into a batch. Which is a terrible idea for me, because now I need to sit and rewrite my queries around it.

turso has implemented their own version of MVCC setup so they support transaction for sqlite but when I looked at Turso’s Golang support, it hadn’t been maintained for 6 to 8 months. This isn’t like Postgres. Here you need a maintained SDK, because they have their own implementation to make it work. For some reason the repo wasn’t updated, so I gave up on the SQLite idea. I felt that for this use case Postgres is good enough, so I went to look for a free Postgres option. I don’t want to self-host Postgres, and everybody who starts vibe coding these days has a bunch of Postgres options like Supabase or Neon, or if somebody is at an enterprise, RDS, stuff like that. I went ahead with Neon, because they have a really good free tier and I’d never exceed it. Neon setup is very straightforward. They even have an agent-friendly setup. You just go set up a Postgres instance, get the credentials and the Postgres URL with the token to authenticate, and start using it, just like any other Postgres database. So I went ahead with Neon, one headache down.

decision 7: security

We have almost all the setup in place now. The only problematic area left is how we secure this whole system. The double-edged sword of serverless is that if someone randomly sends a request to your serverless, it starts running. Even if it fails, or turns out to be some stupid request, you still pay for the incoming request and the compute. So we have to secure our infra here. I’m also a big fan of layered security, where everything is guarded at multiple layers, so that even if someone breaks through one layer, there’s protection at another layer that stops them.

Okay, so the first step: our user interface here, Telegram, talks to our serverless. Fortunately, a Telegram bot lets you set a secret token on its webhook, and Telegram sends that token in a header with every request. Without that token set up in our Golang backend, the request doesn’t even validate. So any incoming request from Telegram needs to carry that token, and only then does our application do anything. In that way, Telegram is secured. But the purpose of the whole design is to be interface agnostic. This is fine as an additional layer of security, but we can’t just rely on that token. There will be a web app tomorrow. There can be Discord, or any custom built app I want to build for myself, and at that point you can’t rely on Telegram’s token, right? The plan is very simple: have some secret and verify it. The very simple approach is: have a token, verify the token, problem solved for now. But you can’t do that check in our main runner.

Because the moment you call the runner, it needs to spin up an instance from the Docker image, run the binary in the image, and only then, when the request comes in, do the validation. That’s just a waste of resources, right? So we need a protection layer before that, a front gate that protects us from such problems, so the request doesn’t even reach our main runner. A simple solution is to add another serverless in front, whose main job is to pull the token from our secret manager and verify it, and only once it’s verified does this serverless talk to our main runner. No other incoming request reaches the runner. That’s one possibility. The problem is, always assume the worst case: a million requests come and hit it, and you have to pay for every request that comes in. Also, everybody has the same problem, right? Somebody has to protect this layer. For every serverless instance, or any instance in general, you’ll have this layer that does your basic authentication. So there has to be a standard solution for it, which is the API gateway.

So forget about the buzzword for a second. Like I said, you need some machine in front of your serverless, and it’s a small machine, because it doesn’t have a lot of work to do. Its only job is to check the incoming secret, and if it matches, forward the request through some private path that only it and your runner share, so your runner doesn’t talk to anybody else. Fortunately Google, like almost all cloud providers, has this implementation called API Gateway, and it does a bit more than authentication (it checks an API key or a JWT before the request goes anywhere). Suppose you have 10 different microservices behind an API gateway, and every user hits the same gateway URL: you can set up rules to route each path to a different service, and do all these sophisticated things with it. For us, since the API Gateway free limit is also really high, the only thing we need is to keep our runner behind the API gateway. Since the API gateway is also Google’s, you can always set up an IAM policy between the API gateway and the runner, so that the runner can only be invoked by the API gateway. That made life easier.

Obviously, just like with all of this, what if you get hit with a bunch of attack traffic on the API gateway? Then you’ll be charged, right? Yeah, that’s obviously there. I think this problem exists universally, even if you have your own machine. It can be handled slightly better with my own instance, because API gateway cost doesn’t depend on throughput, it depends on the number of incoming requests. So I think a VM in front of the runner might be a better approach, because a VM’s I/O is insanely high, you can take the most basic VM, and if you want to scale, you can put a bunch of VMs behind a load balancer doing just the authentication, stuff like that. The API gateway does exactly that behind the scenes. So I just used the API gateway instead of sitting and maintaining all those things for this stupid little software of mine.

So now every request carries an authentication token. In Telegram, apart from the Telegram webhook token, I also have this authentication token, so we have two layers of security: even if the authentication token is compromised, there’s the Telegram-level token as an additional layer. Also, if there’s a web app tomorrow, after OAuth and sign up, I can always go to the secret manager, get another secret, and sign tokens with a private/public key pair. Or do a JWT, that would be better. Yeah, maybe issue a JWT and send it back to the user, so every request carries it and it can be verified. So it opens up a bunch of authentication options. For now, I have this token and I can validate it, and later a JWT that can be validated here, so all good.

And obviously, nothing is baked into the image. I see a few folks adding secrets as part of the image and pushing it to the registry, which is super stupid. None of that here. The code fetches from the secret manager, and everything works that way. And obviously rotation is easy here, so I can rotate whenever I want. At rest, any secret manager is encrypted, so we don’t have to worry about that. The only thing we need to worry about is how we handle the secret after fetching it from the secret manager and using it, and after the lifecycle is done, the serverless instance is dead anyway, so all the secrets are gone too. So we’re good to go.

decision 8: crash

What if the serverless crashes in the middle? What if you give it a 30 minute audio, and for some reason your LLM retries don’t work the way they need to, and it goes to your fallback text to speech model, which partially does the job and still fails to complete it? What are you gonna do now?

There’s also one part we haven’t touched yet. Because it’s Telegram, I was able to send the audio files directly to their endpoint, and they made sure to store it on their side and serve it back in the chat. But when we have an actual app, we can’t do it that way. We need to put the audio in an object store, create a pre-signed URL from it, and load that URL in our website so we can play the audio. So an object store is really important here. Fortunately, Cloudflare R2 has 10 GB of free storage, so I decided to use R2 for this whole purpose.

The audio is generated in chunks, and every once in a while I keep pushing the chunks to the object store, and we keep track in our database of what’s been pushed. It’s a little more than that, but at a high level, all the chunks are in the object store, and the final audio is also stored in the object store. The link to the audio is stored in the database, so when we need a pre-signed URL, we can generate one from the object store. Then we have a retention policy of 7 days, so it just cleans all of it off, and it cleans our database as well, because Neon has a fixed size for me, so there’s no retaining data. In a professional organization, you can’t do hard deletes, because all that data is needed for auditing. You do a soft delete by adding an is_deleted column, and then slowly move the data from one archival tier to another, and finally to something like Glacier-level archival, based on what kind of auditing you need, stuff like that. But that’s not a problem here, so once in a while I check what has crossed the 7 day mark and clean both the Neon database and R2, because I have a very limited free Neon database and I don’t want to exceed that limit.

updated design

decision 9: some cost cutting optimization

Obviously I did a bunch of optimizations to cut cost. The first one is converting a blog into a markdown file, so that I can send that to an LLM instead of sending the link to the blog directly, because the LLM may or may not be able to read it. Extracting content from a website into markdown is straightforward to a certain extent, but I don’t want to sit and do that here. So I used something called Jina Reader, which has good free API access: I send in the URL and get the markdown back. So I didn’t take that headache on myself.

Another thing I noticed: for a blog that’s about 30 minutes worth of narration, if I chunk it beforehand and process the chunks in parallel, I can cut that half hour down tremendously. But I can’t do a lot of parallelism here because of the kind of runner I chose, the most basic one. So for the most basic runner with the most basic spec, I decided to stick with three parallel calls at the same time. I chunk everything, start processing the chunks in parallel, and then use ffmpeg to stitch the chunks together.

It can also fail in the middle, so what if it crashes? Once in a while, I do a kind of hacky sliding window, where I stitch together whatever audio is there, and use the sliding window to keep track of the contiguous chunks and the non-contiguous ones. I generated an image with Claude and attached it below so it’s easier to understand. I keep track of it, so when there’s a crash, the backup is there and it’s easy to restore: just cut the chunk off and continue from wherever it worked. I came up with this random sliding window idea and I think it worked. I deliberately crashed it a couple of times to see if the sliding window chunking works, and it actually saves money, because each TTS call generates a part of the audio. Why waste that audio if you can reuse it and only generate the chunks you don’t have yet? That’s the idea behind it. I think this is one rare time where a LeetCode idea is being used in real life for me, because this is a common LeetCode pattern: there are questions like merge intervals, and then there’s sliding window, and this is like a combination of both lmaoooo

chunking

decision 10: infrastructure as code (pulumi)

Because of all this cost cutting and figuring out which free options are available, we now have a lot of moving pieces, and managing all of them is very difficult. We have a gateway, a secret manager, an object store, a serverless runner, and a database, and each of them has its own policies. Setting things up is very important, tearing down is also very important, and writing a bunch of bash scripts or manually setting everything up is almost impossible. I’d heard about Pulumi a lot but hadn’t used it until now, so I wanted to give it a try. Usually I either write my own bash scripts, which are ugly and not maintainable, or in the worst case I write a little bit of Terraform. But with Pulumi I was able to write it in Golang, which made it even easier. So the whole setup and teardown is basically one single Pulumi project: I just run Pulumi locally and it sets up the whole infrastructure by itself, and teardown is just as easy. The deletion and retention policies are there everywhere, and logging is also added.

As for observability: when you have a server, you store logs as files on the local machine and move them to an object store, or an endpoint exposes the metrics and you scrape them from there. Serverless is a little trickier, because instances come and go, so you need to constantly push the metrics out of the serverless. The way various production systems do it is to run an OpenTelemetry collector or a Grafana stack separately, which the serverless keeps pushing to, and which then pushes into wherever we keep our logs, and then it’s easy, we can export from our log store and stuff like that. But I don’t even want to go into that here, because Google’s default logging system (Cloud Logging, GCP’s equivalent of CloudWatch) is also free. Log level wise, I don’t have a lot of debug logs, it’s mostly info and errors, and since I use it very few times a day, it’s okay for me and comes under the free tier itself. So I’m just using the default here for logging.

pulumi

I deliberately haven’t talked about the code and how the logic flows, because you can obviously figure that out from the use case, and it’s fairly straightforward here. I wanted to write this blog just around the infra and the deployment. I’ve attached the link to the repository below, and I added a bunch of Claude generated design images of what I had in my mind, so you can take a look at those too. It’s deployed now, and I use it every day. So it was fun. There are a few redundant parts in here that I can see are over-engineered and a waste of time, which I’ll slowly keep removing to keep it as minimal as possible. I don’t know what direction this project will go for me, because I’ll only implement a feature when there’s a use case I actually need. Right now it’s very generic, so if I make it a little more specific, I can cut off a few parts. That’s for the future. I can also see this is a lot more focused on backups and stuff like that, which I could relax a little, because I’m the only one using it. That’s also something for the future.

Hope you liked this blog. I’ll see you guys in some other blog. Meanwhile, go check out my GitHub repository for this project. And please send me your mails and DMs about this blog as usual. I’m especially open to hearing your ideas on what can be done better, or your own perspective, and if you want me to clarify something that’s missing here, I can always update the blog.

source code github link