Congrats on the launch. Like a lot of the people right now, am doing some side project with OpenAI + LangChain and immediately got some value out of this, specifically:
* When your chains get long/complex enough in LangChain, it's really hard to understand from debug output what's final prompt that actually being sent, how much it costs, or catching runaway agents. This pretty much solves that for me.
As a "prompt developer", one thing that'd be incredibly useful is a way to see/export all of my prompts and responses over time to help me tune them (basically a "Prompt" button in the left nav).
Congrats on the launch. So nice to see a tool in the space that lets you get up and running in 4 minutes.
Congrats the on launch! I noticed you are referring to the project as open source while using the commons clause, which isn't typically considered an open source license, so I raised this via the discussions of the GitHub repo [1].
Here's another open source alternative: https://github.com/6/openai-caching-proxy-worker
Y'all committed an env file https://github.com/Helicone/helicone/pull/136/files O___O
The entire company is based around the idea of providing metrics for one closed-source platform's API? Or is the "OpenAI application observability" just one part of what the company does? Otherwise it seems like taking the "all eggs in one basket" to the extreme.
Does this run afoul of OpenAI's terms of of service in any way? Using a commercial proxy/broker like this to access their API services instead of using directly.
The reality is that virtually all of the public AI tech available right now is from OpenAI. Over time as more models are commercialised and generally available, it's probable that Helicone will serve them too.
Congrats Justin and team! Excited for you.
Could be cool if you offer quota enforcement. So if something goes rogue on the application side, your layer could not only offer observability but a ceiling to protect against unexpected cost overruns.
Wow so cool! Does this act kind of like a logger and would we be able to have access to the logs later on, or should we bring our own logger as well?
(Also just curious, are you guys just using D1 or KV under the hood?)
What is the market for this solution? Whose pain point(s) are you solving? And what stops OpenAI from "Sherlocking"* you, i.e., making whatever you're building a free included feature and extinguishing the market?
*Or, to use a more modern analogy, "Evernoting"
This is great, however, I am concerned that the various AI 'ecosystem' of all the bolt-on, add-on, plug-ins etc... will be like a billion services all looking for payment - and any complex startup that needs a bunch of these services to build their own service/product/platform, it will be like the current state of streaming services.
So, youll be trying to manage a ton of SLAs, contracts, payment requirements, limits on service access that may be out of your budget to pay for all the various services, API calls, etc.
This is going to be an interesting cluster....
So we need a company thats a single service to access all available AI connects and the multiple billing channels.
However, then you have that as a single POF
So route traffic through you so that you can monetize the insights from the data??
LLM infrastructure is the spot to be right now for startups.
Happy Helicone customer here. It's a dead simple setup. It's great to have the extra charts and logging to debug issues and make sure all is running well.
Congrats to the team!
congrats! Helicone provides one of the biggest missing pieces from the the AI tool dev experience today, thanks for building this and sharing it with the rest of us!
Congrats! We've been happy users of Helicone for the past few months--it literally helped us solve a bug with OpenAI's API where we didn't know why requests were failing and we failed to log some of their responses. Helicone helped us debug that it was a token limit issue really quickly, especially since the logging around hasn't been great.
Love how easy it was to integrate too--just one line to swap out the OpenAI API with theirs.
Perfect! I was just today thinking about how I need to build up my own data set and should be logging all of my transcripts. This is exactly what I wanted.
I want to gather up my chat transcripts, then identify poor experiences in the chat, and then use that to guide fine-tuning. I don't believe that OpenAI actually provides anything to enable this as part of their platform, right?
Awesome.
Is there a consumer version of this?
Like an alternative ChatGPT client or chrome extension that will save my prompts/conversations, tell me which ones I liked more and let me search through them?
From a VC perspective, this sounds like easy money, since it'll probably be acquired by OpenAI or Microsoft if it succeeds just a little.
This is exactly what we needed. We were one day away from an internal build out so the timing couldn't be better.
Hmm, so to integrate I have to basically send my api key to you on every request? Not great
Are you going to be peeking at the data for your own use or is that off limits somehow?
Is data encrypted in such a way so that your company cannot read conversations?
onboarded and started using it in literally 2 minutes. nice
This is awesome guys!
Congrats on the launch!
I have _specifically_ thought of writing something just like this, so it's awesome to see it!
One thing I would really like to be storing with my requests is the template and parameters that created the concrete prompt. (This gets a little confusing with ChatGPT APIs, since the prompt is a sequence of messages.) Custom Properties allow a little metadata, but not a big blob like a template. I see there's a way to have Helicone do the template substitution, but I don't want that, I have very particular templating desires. But I _do_ want to be able to understand how the prompt was constructed. There's probably some danger on the client side that I would send data that did not inform the prompt construction, and balloon storage or cause privacy issues, so there is some danger to this feature.
Backoffs and other rate limiting sounds great. It would be great to put in a maximum cost for a user or app and then have the proxy block the user once that was reached as a kind of firewall of overuse.
Note your homepage doesn't have a <title> tag.