Marco
Author:
đź”— marco_mail
Real discussions and feedbacks of Marco
đź”— Join the Discussion on Forums
App Description
We tried just about every option. Eventually we landed on an extremely robust solution.
Context
When we set out to build Marco, we knew we were committing to two very difficult requirements:
- IMAP-based, not API-based, and
- Cross-platform – web, Mac, Windows, Android, iOS
We believe that full offline support is crucial. “Managing your emails on an airplane with no wifi” is an example use case we frequently come back to. You should be able to read, delete, respond, and organise your emails with no internet connection. When you land and connect to wifi, everything should seamlessly sync.
Building an offline-first app
Here’s a rundown of the initial offline-first options we looked at:
- WatermelonDB
- FOSS, self-hosted
- Database agnostic
- Been around for ages, used in many production applications
- PowerSync
- Not quite FOSS, but has a free Self-Hosted Edition
- Requires Postgres-level integration
- Very complex architecture, requires both changes to Postgres _and_ a separate HA MongoDB deployment cluster
- Appears to have been around quite awhile, but all their case studies are on demo/tiny/side projects
- ElectricSQL
- Looked interesting, but was in the middle of a complete rewrite
- Requires Postgres-level integration
- New version only handles data sync one way – it does not handle mutations
There are many other options, including RxDB, MongoDB Atlas, Couchbase, and on and on. The three listed above are the options that we deeply investigated.We settled on WatermelonDB and built the initial alpha version of Marco on it. The backend implementation is rather simple: there is a “pull” endpoint to GET data, and a “push” endpoint to POST mutations.
We got quite far along with the Marco alpha build, and then had a bit of a panic in November. Our confidence in our WatermelonDB-based offline-first approach was decreasing steadily. We began to seriously question if this technology could actually support a rock-solid, modern user experience.
We decided we needed to find something better.
New Wave of Offline-First
This time around, we threw out any preconceptions we had about Postgres, separation of concerns at the API layer, etc. We had completely open minds and desperately wanted to find the “best” solution, no matter what that might look like. We were now extremely clear on the fact that we had a “tough” offline-first use case, and needed some serious help.
We discovered a host of “new wave” offline first implementations. We talked with the founders/developers of these projects and found so many extremely intelligent and talented people working on what is a very tough problem.
The leaders in this new wave are:
While Triplit and InstantDB can be described as “full stack databases”, Convex is instead an entire backend solution, including API endpoints, etc. For this reason we excluded Convex, as it seems like a huge leap and essentially 100% lock-in.
Why So Many Problems?
There is a saying: if everyone is an asshole, maybe you’re the asshole. Is the problem 5+ offline-first tools, or is it us?
Like with most things, the reality is “a bit of both”. As mentioned, Marco is an incredibly data-intensive application, and from day one, we were pushing these offline-first tools to their absolute limits.
However, we also found the practical limits of these tools to be way lower than one would expect.
What is the underlying cause? In my estimation, the root cause is that all of these offline-first tools for web are essentially hacks. Because of Marco’s web deployment target, which becomes our common denominator, we must support offline-first in a web browser, and web browsers only really support KV storage via IndexedDB.
At this point, we were starting to pull our hair out, and were wondering if we needed to build our own sync engine. Like many others, we’re highly impressed with Linear, but are also aware that their sync engine was a monumental engineering effort.Finally, A
Solution
Some time in early December, I came back across an option which I had glanced over before, but disregarded:Â Replicache.
I think we were initially put off by their strange pricing model and the fact that it’s closed-source.I am so glad we took another look. In terms of backend implementation, Replicache is somewhat similar to WatermelonDB, in that you need to implement push and pull endpoints in your backend, and it is otherwise entirely database-agnostic.
The frontend is where the crucial difference lies – Replicache is just a KV store. It is a thin layer on top of IndexedDB that adds reactivity and some querying DX. That’s it. You get rawÂ
get andÂset performance.Some performance benchmarks are outlined here. The perf is truly remarkable.
Future of Offline-First
We embarked on a long and rambling journey through essentially all prior art and work in the offline-first world. It’s a very hard problem to solve.
The good news is that there are many new teams and projects actively and energetically working on this problem.Imagine a world where, as a fullstack developer, you can read and write data from an SDK in both your backend and your frontend, and they magically sync with each other.
All your apps are instantly responsive. All your apps work offline out of the box.
This is already possible today with Triplit or InstantDB, if your use case is reasonable. And things are only improving.
I believe 2025 will be a year where HTTP/REST APIs will start to feel antiquated. Don’t share endpoints – share databases.
Project Overview
The post details the journey of building an offline-first email client named Marco, emphasizing the challenges and solutions found in achieving full offline support across multiple platforms. The team explored various offline-first tools, initially settling on WatermelonDB before transitioning to Replicache for its superior performance and simplicity. The narrative highlights the complexities of offline-first development and the ongoing evolution of tools to address these challenges.
Features & Benefits
âś… Cross-platform support
âś… Full offline functionality
âś… Use of robust solutions like Replicache
Areas for Improvement
🔄 Complexity in achieving full offline support
🔄 Dependency on evolving offline-first tools