Hi HN! We're Lucas and Lucas from Briefer and we're building better notebooks.
Our notebooks are kind of a mix between Notion and Jupyter with extra features, like the ability to schedule notebooks, turn them into dashboards and apps, and write SQL queries whose results turn into data frames automatically.
We're building better notebooks because we think they're a great idea poorly executed - for three reasons.
The first problem with notebooks is that they're difficult to share. Non-technical people don't want to download docker containers and install Python libraries to see what the data team is doing. Then, the data team takes screenshots of their work and pastes them somewhere else. The issue with this approach is that the data gets stale, and the output is not interactive, so it's difficult to get feedback and iterate.
The second problem with notebooks is that they get too messy too quickly. One morning you have 10 blocks, and by the end of the day you have 192, but only six of them are meant to be seen by other people and the rest is just you jiu-jitsuing with the data. Consequently, even if non-technical people could see your work, they'd have a hard time figuring out what's happening.
The third problem with notebooks is that it takes too much work to do simple things like when you want to query a database. In that case, you need to have the database credentials in your machine, and you have to write a bunch of wrapper code. Calling APIs, plotting simple graphs, and adding interactive components is equally as annoying.
We're solving the sharing problem by bringing notebooks to the cloud (so you can schedule them) and using CRDTs to manage their state. Whenever you edit a Briefer notebook, we reconcile that using Yjs, and then propagate it to everyone else who's editing that notebook. Regarding compute instances, each workspace gets its own, and we provision them on demand.
By the way, we manage the execution state of your blocks using Yjs too, which makes it much more stable and responsive across all clients. When you click "run", for example, we don't immediately send a request to run the block. Instead, we change the block's state to "execution-requested". Then, the observers in the back-end react to the change and update the block's state with the results. In other words, the front end and the server communicate with each other through the notebook's state. One side updates the state, and the other reacts.
To solve the "messiness" issue, we've invested a lot of time in designing clean notebooks and allowing you to organize blocks in ways that make them more presentable. In Briefer, you can group multiple blocks into tabs and decide which blocks appear in the published version of your notebook. That way, you can hide all that data wrangling and focus on results. We also know that a vertical format is not always the best way to display results, so you can use your notebook's outputs to build dashboards too.
Last but not least, we're reducing friction in simple tasks like plotting graphs and querying databases. In Briefer, you can plot graphs using a click-through interface, and if you need to plot something more intricate like a Sankey chart there's also a built-in AI assistant that you can prompt to generate code. When it comes to queries, we turn their results into dataframes automatically, and you can use SQL to query uploaded files too (we use DuckDB for that).
We're really happy we get to show this to you all, thank you for reading about it! Please let us know your thoughts and questions in the comments.
1. product managers will try to make things dead easy, by setting up simple excel formulas & charts. Taking screenshots etc
2. data engineers will do more technical but opensourced solutions like superset & dbt
3. corporates usually tends to build the end UI themselves. I remember sth that (maybe?) Pinterest opensourced.
I deeply understand how hard it is to build a notebook like this. And there are indeed some problems lying in the collaboration and UX.
It's just so hard to propose those features, given the problem is usually the data itself and a highly competitive market. Popsql does not thrive as I remembered.
reply