Post It Notes will do if you have a good system.

Those who don't understand Unix are condemned to reinvent it, poorly. - Henry Spencer .. via https://github.com/globalcitizen/taoup

S3 is strongly consistent, if you need it anyways you can just use s3 keys to deconflict and store the workflow state.

The suitability of Litestream for production disaster recovery is also an open question in my mind. I used 0.3.x for several years and when I tried to upgrade to the 0.5.x series there were runaway disk usage problems that would have caused downtime had they made it to prod. As far as I can tell these have not been entirely addressed, although recent bug reports suggest that they might be getting closer.

I want to love it, and I don't take open source projects like this for granted. But during my last production upgrade I chose to decommission Litestream in favor of a dumber, less granular solution using sqlite3_rsync and nightly backups because there is no point in using a backup system that is not rock solid.

Postgres also does not synchronously replicate for free. You can setup both to get a confirmation write if you require that durability.

  synchronous_commit = on

  PRAGMA synchronous = full

"Durable workflows without the durability"

That's distributed workflows :)

i mean it's durable as long as nothing crashes or litestream has a data corruption bug which only happens every other release...

try setting up replication/failover in postgres, it's much more work.

Same. The prompt was essentially, every checkbox in this PLAN.md should be task in SQLite.

What would be the main differences between DBOS and Obelisk?

Could you give an example of a case where you'd use SQLite instead of jq or grep through Markdown?

This reads like an advertisement for Temporal :)

Interesting about the files vs db approach. I have been going back and fourth. I landed on db as well.

Word on HN is that you're either paying more money than you expected for temporal's managed solution or taking on substantial ops burden ultimately running their very heavy system yourself.

I wouldn't know, I've not done either, but I'd like to learn more from your or other's experience.

It sounds like you’re running this mostly on a single machine? Temporal gets much more complex with scale. Cassandra isn’t fun to manage. Ringpop and TChannel are hard to debug when things go wrong. The SQL backend support doesn’t support horizontally scaled replicas (just single instance) due to consistency requirements. Depending on how your code is written, modifying code baked into workflows becomes complex, as anything that modifies the history event ordering breaks determinism in already-deployed workers.

We use it heavily and everyone who started on it doing simple scripting/automation all love it, everyone who built real production systems on top of it all hate it. Possibly operator error, but my experience hasn’t matched the rosy picture painted in these comments.

My current client has a forest of 90+ SnapLogic pipelines that were badly written and maintained even worse; one of those was completely wrong, in that it generated wrong accounting data which could eventually have financial, fiducial and legal repercussions.

I rewrote the pipeline in Python (a correct version of it) with state management in SQLite and logs in plain old flat files, and everything has been running smoothly ever since. In fact this is the only data flow that has worked without errors or interruptions in the last six months.

Instead of replicating the db file with Litestream I do a remote backup with Restic before and after each run; it's not an exact replacement of Litestream as we could possibly lose a whole run if the machine died / disappeared at the end of a run, but it lets one restore any day very easily. In an ideal world I think we should have both (live replica + backups).

> It saves a lot of tokens when an agent can just query a specific row instead of having to fire up jq or grep through markdown

Just wanted to make sure no one missed this point in your comment because eventually users will be paying the full cost for tokens instead of VC's paying, with GitHub Copilot's pricing realignment leading the way.

That's a crazy story. National TLD is a weird business from the beginning.

> SQLite is surprisingly performant for single node applications even when comparing to Postgres.

In the context of SQLite being understood to be a quite excellent piece of software - shouldn't we expect it to be?

In the context of a single-node, Postgres is overkill. It should not be expected to be competitive with SQLite.

This is almost like benchmarking an in-memory HashMap to Redis and being surprised that it performs well in ideal conditions.

When used properly, SQLite is effectively an in-process method invoke. If the only remaining things in the way are your runtime, kernel, file system and a local NVMe storage device, you may find it massively outperforms hosted alternatives.

Leaving the current thread is where you lose the game in terms of latency. SQLite can work on timescales measured in microseconds if you don't force interthread communication.

Well if you run a tiny single-threaded app then SQLite is a nice simplification over spinning up a separate machine for Postgres.

It's almost as if Postgres isn't perfect, and one size shoe doesn't fit all.

Some people want some of the benefits you get from SQLite.

SQLite is obviously not perfect, but it's an incredible piece of software, and people regularly find good ways to make use of an excellent pieces of software.

I absolutely 100% do not understand it either. At all. Every time I try to over the last year or two I come away with the conclusion its something that sounds cool (to me too!) but is guaranteed to cause more problems than more obvious solutions.

That being said I'd kill for someone who used it and benefited to explain it to me in a practical sense. (specifically where syncing is involved, and syncing a subset of the SQLite is necessary. If it's "just" a document store thats treated like a blob for syncing/backup, that's familiar. If it's all in one storage but only local, that's familiar.)

Re: TFA, I guess it would have helped if I knew what Obelisk was, which is on me, and a more in-depth explanation of how this ties into AI/agents, which is on the industry/writer.

So teach them. If you want to bring up computer science fundamentals, the question is where does SQLite sit with regards to the CAP theorem. Consistency, Availability, and Partition tolerance. SQLite isn't a distributed system, so there are no partitions to tolerate, so it's a CA system. Other databases make different tradeoffs. For systems that don't need concurrent writes, SQLite is pretty great! There are no users to manage, no permissions, no daemon to run, no server and port to mix up. Just open a file on disk using a library.

Isn't concurrency also limited by your machines disk speed for writes, what difference does it make if you write sequentially vs concurrently? Why does concurrency even matter for databases?

sqlite is more like a file format than a database. it competes with .xlsx.

> "SQLite for everything" crowd is a little bit inexperienced.

every time i see it in a real application, it becomes a huge focus of issues (for example: jellyfin, hermes, openwebui, comfyui)

> This is a foundational principle of computer science

How exactly is this a foundational principle of computer science?

You seem to have a rather limited understanding of what kinds of concurrency exist and how those needs are best met. Whether something is a server or not is not very relevant to this discussion.

SQLite is an excellent production db for many real world workloads, as has been widely documented. It is very different to Postgres, so requires learning a whole new skill set.

One way to think about it is that SQLite can work well for the parts of your system where there is naturally strong partitioning.

Computer science no more get its hands dirty with concrete software than physics primarily being about building bridges.

It is not «a foundational principle of computer science».

I had very good results giving 1 SQL DB per go routine, so the accesses were serialized up front, on a very high volume (130K requests/second) service. Exact transactionality was not a product goal, and the SQLite was just to backup the in memory state. If we lost a little due to abend or something, that was ok (although for normal maintenance it caught SIGTERM and stopped the listen and then waited for in flight calls and then flushed the remaining changes to SQLite; then on startup it would read the SQLite into memory to populate before taking the listen; persistent storage across container runs, and never both reads and writes to the same file at the same time. (It also just closed the DB and opened a new one when it hit some limit of rows, so as not to fill the disk; the max size of the SQLite corresponded to the max size of the LRU map being served from in memory; then it just flipped A / B between "a full memory worth of data stored" and "the currently updating state." A lot easier than having to write out proto bufs to disk or whatever I would have done for transient (during restarts/maintenance) persistence.

That’s why there are billions of SQLite databases right?

SQLite is likely used more than all other database engines combined. Billions and billions of copies of SQLite exist in the wild. SQLite is found in:

Every Android device Every iPhone and iOS device Every Mac Every Windows 10/11 installation Every Firefox, Chrome, and Safari web browser Every instance of Skype Every instance of iTunes Every Dropbox client Every TurboTax and QuickBooks PHP and Python Most television sets and set-top cable boxes Most automotive multimedia systems Countless millions of other applications

https://sqlite.org/mostdeployed.html

There are many cases where SQLite + concurrent front end (like a go net/http server) can handle all the load that a service might ever conceivably have to handle, especially if allowed to scale up hardware over time. You can trivially scale up SQLite to, what, hundreds of thousands of tps?

The only thing you really give up is HA/failover and DR. But there are solutions to deal with those. And single-server systems are generally surprisingly robust (since, in the absence of very complex control planes, uptime goes down with more systems).

I worked on an app that had sqlite databases per user... it was fine.

Sqlite is good for lots of stuff, but you're probably focusing your days on high-scale webapps that want sharding with networked DBs. That's one domain, and an interesting one, but there are lots of others.

I'm a big fan of re-evaluating prior "best practices" in light of technology changes, especially in ways that improve simplicity. Running my family's social media site off a single sqlite DB on a VPS is great. ~15 users, almost zero maintenance. I run my FreshRSS instance off of sqlite, as well as my "now" page. At work, I used sqlite for all kinds of things over the past decades: as an ad hoc job queue, as a quick way to ingest and query lots of logs locally, and present/filter in realtime with simonw's excellent https://github.com/simonw/datasette.

I don't think it's every "sqlite for everything" as much as it is "sqlite in lots of places you probably didn't think to apply it."

kentonv/Cloudflare's work on sqlite at the edge might have made this thinking a bit more popular, but it was always around. https://blog.cloudflare.com/sqlite-in-durable-objects/

I suspect being aware of all those little neat cases and wanting to leverage sqlite for them may be an indicator of experience, rather than the opposite.

I mean - I agree for the typical multi-user, SaaS webapp. But I don't think that's what these folks are proposing. If they are - yeesh, count me out.

If on the other hand they're talking about single-user, software in the small - hell yeah. In fact, I'd also promote DuckDB in this regard (mostly for analytics) - with the power of a single machine these days, you can do a surprising amount and never have to worry about distribution. Unless you know you'll have to, in which case you're probably just digging yourself a hole?

Sure, SQLite doesn't solve every problem -- but in many cases it solves the need at hand with the reward of one less piece of infra required to support it.

I see obsessions with tooling/solutions constantly from experienced devs who fall in love with the original solution and think it's the only way to do things -- so the experience part cuts both ways.

How many production apps do you think have enough users to justify these huge DB servers?

Are you one of my enterprise customers? What if your workload does not require write concurrency?

And of course there are now several responses proving your point.

Scale to zero is very useful.

I think you'd be surprised to learn how many real production apps are actually running on top of SQLite (by way of Cloudflare D1).

Thing is SQLite scales better than both those network databases [1] if you're prepared to stick with one big machine (+ a standby).

This is even more obvious when you start doing transactions processing an row locks across the network limit you to 1-3k TPS that you cannot scale out of (Pareto distribution is merciless).

[1] - https://andersmurphy.com/2025/12/02/100000-tps-over-a-billio...

Someone with experience would know that concurrency isn't a universal requirement.

If your data is naturally sharded (users) with writes happening within a single shard, parallelism becomes easy. The request is routed to the shard hosting the user's data and reads/writes locally.

This makes scalability _much_ easier to reason about. It's cut-paste, cut-paste. Every N users needs another shard.

It does buy you a _different_ set of problems, like cross-shard querying (analytics) and how to do load leveling as users age out.

But it avoids the whole shared index scaling problems from inserts/updates with large user counts.

It becomes a hierarchical instead of a relational database.

there is a difference between concurrency in a distributed environment and concurrency on a single machine across processes. SQLite is incredibly useful for the latter.

you seem like the inexperienced one to me..

For me, I have a use case that needs to support a few thousand users, probably a few hundred concurrently.

The combination of SQLite (libsql, a concurrent implementation of sqlite) and Rust means I can do so from a $2/m VPS and a single server instance.

Backups are done via a cron job that uploads to S3.

Does it pass the "Netflix scale" test? No

But it doesn't need to. I'm not profiting from the service and SQLite offers a path to scale if/when ready because... well it's just SQL and I can literally just swap `libsql::Connection` with `psql::Connection` in my repositories.

Plus upgrading from a $2/m VPS to a $10/month VPS quadripples the number of concurrent users I can support.

IMO, you can vertically scale extraordinary far with SQlite and an efficient server implementation.

I'd wager that 90% of forum websites, wordpress sites and online shops would be fine with SQLite.

There is something appealing about "it's just a file" (it really isn't; it has locks and a WAL), but I agree with you.

I think people are afraid to read the documentation for postgres. You can start it up in milliseconds. Fast enough and light enough to run one copy for every test case in your test suite, or whatever you're using it for. (mkdir /tmp/whatever; initdb -D /tmp/whatever --no-instructions -A reject -c listen_addresses= --auth-local=trust --no-sync -c fsync=off -c unix_socket_directories=/tmp/whatever -U postgres --no-locale; postgres -D /tmp/whatever) Now you have a test database that behaves exactly like production because it's exactly like production. (OK, turning fsync off makes it a lot faster than production, so be careful.)

Personally I like Postgres for this reason too. Its extremely easy to run with Docker, I can dump data from all kinds of apps in there and I know it's not going to take any rearchitecting as soon as I need multiple concurrent writers.

I think docker is still super underappreciated so setting up any kind of server is seen as a chore. In my eyes it makes running tons of services like this very easy, so ill take the extra functionality, extensibility etc of postgres.

And I don't understand the obsession with server-based databases for single apps. Especially in containerised setups, every "app" gets its own database anyways, and if the app is further broken down into services, they usually communicate between each other and not with a shared database. So in those cases, what do you gain by pulling the database out of the "process" and onto the other end of a socket? In most cases, absolutely nothing. So why bother?

Don't get me wrong, I've worked with plenty of server-based databases, including proper dedicated database servers. It's great tech and often the best tool for the job. But not always and I'd argue not in the majority of uses.

Most apps do not actually need the concurrency capacity that Postgres or MySQL are designed for.

> SQLite for everything

is just wrong, and I don't think that the SQLite fans are that crowd. Taking a database server for everything is probably possible, but often unnecessary. With experience, one can properly judge when SQLite is sufficient and when it is not.

So arguing that the SQLite crowd is inexperienced feels weird, because inexperienced people have a much harder time judging when to use what and can just use the database server all the time (even when it is overkill).

> SQLite is an embedded database

Yes, but that's not its main selling point. An SQLite database is also a single file, which makes it incredibly easy to replicate, backup, transfer, restore, etc.

I think the SQLite website itself says it best:

> SQLite does not compete with client/server databases. SQLite competes with fopen().

It's touted by the people who use the word "just" a lot.

"Just use postgres" "Just use sqlite" "Juse use a monolith" "Just use sftp" "Just use an ec2 instance"

Usually these people have flunked out of the school of (distributed system) hard knocks. They couldn't hack it and are retreating to familiar.

The funny part is when one of those people fluke themselves into senior management when their saas takes off.

Inevitably they have to suck it up and hire experts in the same technologies that "no one needs".

SQLite also gets really slow at around 50 million rows.

Probably going to get some winces for this but I do everything with flat files. Maybe my data aren't massive enough, but I mean I can do the relational thing by just having these metadata in some column, and returning rows that contain my desired information in these columns. Even if the file were too big to fit into memory one could just subset chunks of it and chew through. All this can be done with no dependencies, just base libraries of a lot of languages.

Is this bug logged?

I've been following litestream for a while, and it seems like the project has been hijacked by a vibe coder. I wouldn't trust it for critical tasks anymore.

Surfacing for the thread:

Armin Ronacher's "Absurd Workflows: Durable Execution With Just Postgres" https://lucumr.pocoo.org/2025/11/3/absurd-workflows/

Are logs all you need for durable workflows? I'm confused here. How'd persist and query nested or related data over logs? By logs I assume you mean something like elasticsearch or meilisearch?

Shortly followed by:

"Sockets are all you need for durable workflows" and then finally "Kernel primitives are all you need for durable workflows."

But seriously, part of being a professional is using the right tool for the job.

In all seriousness, I’d take a “s3 is all you need for durable workflows” and use it in data processing applications that move data from s3 -> s3 with no other dependencies.

Wait no further. It's already happening.

One reason why a "logs are all you need" solution may fail: untrusted-log-as-injection[1].

Check those SBOM, and don't forget to include their CICD pipelines[2].

[1] https://news.ycombinator.com/item?id=48315440

[2] https://github.com/jqwik-team/jqwik/issues/708#issuecomment-...

Yep. But we all know that one machine can and will fail (or be patched and restarted), so the log needs to be distributed.

Different workflows should probably go in different buckets or "topics" for clarity. Since it's distributed, the system must guarantee that the log items are stored in the same ordering ("offsets") among the nodes.

Not a bad way to do things.

Pardon my ignorance trying to follow up on what is most likely sarcasm but is this not Kafka's claim to fame?

I am joining a new project and need to know to what extent Kafka is still a part of the future for new big data projects. It doesn't seem like there are alternatives at the high end but instead the question is when other technologies (that are easier to manage, require less compute, etc.) max out.

Hello, DuckDB devrel here. First, thanks for the kind words :)

Second, it's funny you should mention the 20-line awk script. I was making a very similar argument yesterday at the Ubuntu Summit: at some point, using shell scripts with GNU coreutilus becomes impractical, while DuckDB SQL scripts scale better in terms of complexity and maintainability (and often also performance). My slides are here: https://blobs.duckdb.org/slides/duckdb-ubuntu-summit-2026.pd... (pages 32 to 36)

Third, MotherDuck develops a closed-source DBaaS on DuckDB. They build on DuckDB, and you connect to MotherDuck with DuckDB but they are a separate VC-funded company headquartered in Seattle. DuckDB is developed by DuckLabs, a bootstrapped (revenue-funded) company in Amsterdam. And the IP of the project is in a third organization: a Dutch non-profit called the DuckDB Foundation. For details, see https://duckdb.org/faq#how-are-duckdb-the-duckdb-foundation-...

I use DuckDB and like it. Since many mentioned GB level json in this post, so they have large amount of data. Been column based, DuckDB uses more RAM as row count grows. It can be an advantage or disadvantage depends whether memory is constrained. Traditional row based DB such as SQLite can deal with large database with less memory.

Litestream is just the replication layer, it works with any S3 compatible storage, and all of these in their guide as well https://litestream.io/guides/#replica-guides

My version of this is the “N+1” principle. Build for one more foreseeable use case than you currently have. The domain model will click in when you need to generalize a solution, and you’ll gain the ability to see your particular solution as one of several to the problem, and thus evaluate fit and tradeoffs more clearly.

Don’t do N+2. The goal isn’t to predict the future, nobody can do that. The goal is a durable understanding of the domain and the best fit implementation you can get with that current understanding and resources.

That said, SQLite passes that bar for me in most use cases.

Yes, this is basically my only issue with SQLite. SQLite with a strict type system would be great.

it's a single file.

You can use strict tables: https://sqlite.org/stricttables.html

  create table events (
    id integer primary key,
    name text not null,
    event_date text not null check (
      -- YYYY-MM-DD
      event_date glob '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
      and date(event_date) is not null
      and date(event_date) = event_date
    )
  );

  sqlite3.IntegrityError: CHECK constraint failed:
    event_date glob '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'

Read their docs

It gets praise because of stuff other than the type system.

I agree it is disappointing, especially before strict tables.

You should check out DuckDB which is basically SQLite but with proper types. Although it is also OLAP (struct of arrays) rather than OLTP (array of structs) which may have worse performance for typical SQLite loads. In practice I doubt it matters if you have an application where you're considering either.

This is the fault/price of backwards compatibility. Most users of SQLite should just fire off a few pragmas on each connection:

    PRAGMA journal_mode = WAL
    PRAGMA foreign_keys = ON
    # Something non-null
    PRAGMA busy_timeout = 1000
    # This is fine for most applications, but see the manual
    PRAGMA synchronous = NORMAL
    # If you use it as a file format
    PRAGMA trusted_schema = OFF

You might need additional options, depending on the binding. E.g. Python applications should not use the defaults of the sqlite3 module, which are simply wrong (with no alternative except out-of-stdlib bindings pre-3.12): https://docs.python.org/3/library/sqlite3.html#transaction-c...

Also use strict tables. https://www.sqlite.org/stricttables.html

While it has bad ergonomics, you can also use CHECK constraints. For example, using sqlite's built in date support, it's possible but awkward:

    CHECK (
      date(my_date_col) IS NOT NULL
      AND my_date_col = date(my_date_col)
    )

The IS NOT NULL is needed because date returns NULL for invalid dates; the other check because it also accepts Julian days (date('2026') is sometime during year 4707 BC).

They've been improving that recently:

2026-04-09 (3.53.0) - "Enhance ALTER TABLE to permit adding and removing NOT NULL and CHECK constraints"

I use my own sqlite-utils CLI/Python library to work around these limitations: https://sqlite-utils.datasette.io/en/stable/python-api.html#...

it does cost network stack

This is exactly what I had dreamed of except without being aware of the session extension... I put it off because I thought I would have to create a layer opening sequential partial sqlite dbs and somehow filtering to the most recent version of each record in-between compactions.

It sounds like sqlite sessions handles the hardest part and you've sanded down some of the rough edges while implementing the glue bits for s3 (generously licensed MIT); thanks for the heads-up!

I rolled my own uptime but how do you vary region or do residential testing ?

I am doing approximately the same for some of these as well, plus CRM. For me, CRM is just a list of contacts with a state and some other fields, plus record who was contacted and when. But I can directly pull information that is relevant to me. Integrating to an external CRM is more work and arguably more risky than rolling your own.

Thanks for your words of wisdom. I share same sentiment of simple apps. I self-learned coding thru 1990s and I am sick of herding distributed overgrown software at work.

Quick question: how do you inspect database contents of your deployed (micro)apps? I used SQLite in one production app and what I didn't like was to use terminal sqlite client on the server or having to copy DB to my laptop to query it with sqlitebrowser. With my Postgres server it is much simpler to just query by rich GUI client with SSH tunnel.

This is the way, with "bus factor" as the one downside.

What happens when you get hit by a bus or someone with higher annual revenue tracks you down because of this comment and hires you away from this custom software stack with a bigger slice of the profits?

It sounds like there's also a bus factor for your server but I'm sure you're aware of that though it sounds like your clients aren't.

Open source it all, I double dog dare you

I'm also a Go+SQlitemaxxer. For my first SaaS I focused on keeping the list of external services very short, as I basically couldn't afford it. I only picked those services that were a) vital b) could be replaced with a homegrowned/FOSS alternative.

Besides keeping the costs down, I find that this approach makes it way more enjoyable to build and easier to manage than having 10 services/subscriptions.

*PS: I think that you could add a small QR Code for the iOS/Apple installation on your app's website.

Thank you for sharing, if you don't mind can you share some Go+SQLite leanings as someone actually pulling this off?

1) How do you do backups? Do you use github.com/benbjohnson/litestream? CRON job backup with rsync?

2) Any issues with large databases and many clients? Is there a TPS or DB size where SQLite becomes problematic?

3) How do you deploy new binaries and safely shutdown the old instance? Caddy change to route to new binary + Go's HTTP server graceful-shutdown on old instance?

4) Do you use a pure-Go SQLite lib or one of the CGO libs?

sorry about that, I do love Beatles!

Persistence in folly leads a fool to enlightenment.

Yup, when I look back at the silly stuff I did when I was somewhere in the middle (CQRS + event sourcing I’m looking at you), it’s interesting.

It is a source of expertise, because you really learn a lot from it. But when you become old (43 over here), you really learn to appreciate “boring” solutions.

Check out https://obeli.sk/blog/comparing-dbos-part-1/