Supabase Storage now supports the S3 protocol
This is great news. Now I can utilize any CDN provider that supports S3. Like bunny.net [1] which has image optimization, just like Supabase does but with better pricing and features.
I have been developing with Supabase past two months. I would say there are still some rough corners in general and some basic features missing. Example Supabase storage has no direct support for metadata [2][3].
Overall I like the launch week and development they are doing. But more attention to basic features and little details would be needed because implementing workarounds for basic stuff is not ideal.
[1] https://bunny.net/ [2] https://github.com/orgs/supabase/discussions/5479 [3] https://github.com/supabase/storage/issues/439
Here is the example of the DuckDB querying parquet files directly from Storage because it supports the S3 protocol now - https://github.com/TylerHillery/supabase-storage-duckdb-demo
I like to Lob my BLOBs into PG's storage. You need that 1-2TB of RDS storage for the IOPS anyway; might as well fill it up.
Large object crew, who's with me?!
This looks great! How easy is it to self host Supabase? Is it more like "we're open-source, but good luck getting this deployed!", or can someone really build on Supabase and if things get a little too expensive it's easy enough to self-host the whole thing and just switch over? I wonder if people are doing that.
Always thought it’s kind of odd how the proprietary API of AWS S3 became sort of the de-facto industry standard
Supabase also announced this week Oriole (the team not just the table storage plugin) is joining them so I guess this is part of the same story. Anyway it's nice timing I was thinking about a hookup to Cloudflare R2 for something and this may be the way.
Dear supabase. Please don’t get bought out by anyone and ruined. I’ve built too many websites with a supabase backend now to go back.
A question about implementation, is the data really stored in a Postgres database? Do you support transactional updates like atomically updating two files at once?
Is there a Postgres storage backend optimized for storing large files?
What are the chances of Supabase doing a license change? Seems to be fashionable these days so always a little wary of building on these sort of platforms
This is really nice to see! I asked about that feature almost 2 years ago as we wanted to use Supabase for everything. Unfortunately there were no plans back then to support it, so we had to use another provider for object storage.
Congrats on the release!
Is there a formal s3 protocol spec or do these companies try to reverse engineer/feature match what AWS provides?
The article does not mention: do you support pre-signed URLs?
https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-...
You specifically say "for large files". What's your bandwidth and latency like for small files (e.g. 20-20480 bytes), and how does it compare to raw S3's bandwidth and latency for small files?
I tried to migrate from Firebase once and it wasn't really straightforward and decided against doing it, I think you guys (if you haven't already) should make migration plugins a first class priority that "just works" as the amount of real revenue generating production projects on Firebase and similar are of a much higher number. It's a no-brainer that many of them may want to switch if it were safe and simple to do so.
Shouldn't it be API rather than protocol?
Also my sympathies for having to support the so-called "S3 standard/protocol".
I wish supabase had more default integrations with CDNs, transactional email services and domain registrars.
I'd happily pay a 50% markup for the sake of having everything in one place.
Supabase is great and I’ve used it for a number of projects over the years both with a backend alongside it or direct from the client with RLS.
There are some weird edges (well really just faff) around auth with the JS library but if nothing else they are by far the cheapest hosted SQL offering I can find so any faff you don’t want to deal with there’s an excellent database right there to allow you to roll your own (assuming you have a backend server alongside it)
Is there any request pricing (I could not find a mention to it on the pricing page). Could be quite compelling for some use-cases if request pricing is free.
I just finished implementing S3 file upload in nextjs to cloudflare R2 with a supabase backend. Wish I had been lazy and waited a day!
One of the big wins we get from AWS is that you can do things like load structured data files (csv, parquet) from S3 directly in Redshift using SQL queries.
https://docs.aws.amazon.com/redshift/latest/dg/t_loading-tab...
Very cool and very good timing. We just released better support to directly query S3[0] in Definite[1]. It's powered by duckdb.
Hi, a question, but first some background. I've been looking at solutions to store columnar data with versioning, essentially Parquet. But, I'd also like to store PDFs, CSVs, images, and such for our ML workflows. I wonder if now, that Supabase is getting better for data science DuckDB crowd, could Supabase be that one solution for all this?
Do you think Supabase Storage (now or in the future) could be an attractive standalone S3 provider as an alternative to e.g MinIO?
Just commenting to say I really appreciate your business model. Whereas most businesses actively seek to build moats to maintain competitive advantage and locking people in, actions like this paint a different picture of Supabase. I'll be swapping out the API my app uses for Supabase storage to switch it to an S3 API this weekend in case I ever need to switch it.
My only real qualm at this point is mapping JS entities using the JS DB API makes it hard to use camelcase field names due to PG reasons I can't recall. I'm not sure what a fix for that would look like.
Keep up the good work.
This is great news, and I agree with everyone in the thread - Supabase is a great product.
Does this mean that Supabase (via S3 protocol) supports file download streaming using an API now?
As far as I know, it was not achievable before and the only solution was to create a signed URL and stream using HTTP.
no feedback on this in particular, but I love supabase. I use it for several projects and it's been great.
I was hesitant to use triggers and PG functions initially, but after I got my migrations sorted out, it's been pretty awesome.
At $0.1/GB of egress it’s not super attractive compared to B2 or R2 for anything but trivial projects.
I wish they would offer a plan with just the pg database.
Any news on pricing of Fly PG?
Can Supabase host static content yet (in a decent way)?
In this setup, can postgresql query data stored in object storage? i.e. Hive/Icebarg table.
Is iOS support a priority for supabase?
Do you support S3 event notifications?
Now we just need flutterflow to get off the firebase bandwagon.
Friendly reminder that Supabase is really cool, and if you haven't tried it out you should do it (everything can be self hosted and they have generous free tiers!)
Cool to see that Supabase is adding S3 protocol! Nice to see more and more storage solutions available.
We, at Milvus, I've integrated S3, Parquet and other ones to make it possible for developers to use their data no matter what they use.
For those who have used both, how do you find the performance and ease of integration compares between Supabase and other solutions like Milvus that have had these features for some time?
hey hn, supabase ceo here
For background: we have a storage product for large files (like photos, videos, etc). The storage paths are mapped into your Postgres database so that you can create per-user access rules (using Postgres RLS)
This update adds S3 compatibility, which means that you can use it with thousands of tools that already support the protocol.
I'm also pretty excited about the possibilities for data scientists/engineers. We can do neat things like dump postgres tables in to Storage (parquet) and you can connect DuckDB/Clickhouse directly to them. We have a few ideas that we'll experiment with to make this easy
Let us know if you have any questions - the engineers will also monitor the discussion