Oh yes for Claude I use LiteLLM as a proxy to use it with OpenWebUI.
I'll try librechat too (never heard of it before) but I wonder if it has the same capabilities like voice and python tools. And ollama support (95% of my AI interactions are running locally)
I think that's probably the shim I was referring to - it has hardcoded context length, but it is either implemented incorrectly, Anthropic ignores it, or maybe it's on openwebui to manage the window and it just isn't? Not sure. I found it kept getting slow, so I was starting new conversations to work around that. Eventually I got suspicious and checked - I'd burned through almost $100 within a few hours.
LibreChat isn't as nice in some areas, but it's much more efficient in this regard.
I do exactly this, use LiteLLM to bridge it. In fact I use LiteLLM to bridge OpenAI and Groq too. Even though OpenWebUI supports them directly, with LiteLLM I can control better which models I see. Otherwise my model list gets cluttered up. I configured this back when OpenWebUI only supported one OpenAI endpoint but I kept using it because it's just quite handy.
And no it doesn't cost extra credits, isn't ignored and doesn't have hardcoded context length. It works perfectly.
Also, it's pretty easy to find unresolved bugs related to openwebui not handling context length parameters correctly - I believe I actually read something from the author saying that this parameter is effectively disabled (for non-local LLMs maybe?).
Needs some usability testing - I suspect even five minutes of watching a non-technical user trying to use it would be very illuminating. UI is unintuitive throughout. Consider, one of the most frequent actions you will ever do in this type of app is start a new conversation, and here it is represented by a little button near the bottom of the sidebar where you might look for 'Settings' - apparently collapsing the sidebar is much more important. Tiny text, low contrast text, confusing collapsing/expanding sections, lack of whitespace and/or colour to differentiate message pair halves, confusing hover actions for copy, no floating or bottom of code block copy button... I'm pretty sure configuring API keys was weird for some reason as well, but I can't remember what it was.
I had to think about this for a minute because the abstraction levels idea is very appealing, but ultimately I think this comparison is specious and misses the core issue entirely.
XY problem is a communication and problem-solving anti-pattern where someone has already decided on an approach and is just asking about implementation details, rather than being open to potentially better solutions that might emerge if they just explained wtf they are trying to do first. Framing it as different abstraction levels obscures this real issue and incorrectly suggests it's just a natural variation in thinking styles rather than a specific mistake in problem-solving approach.
Season 3 is so great it easily eclipses the first two, despite my pretty strong nostalgia-bias. It's like the first two were just a warm-up - and we needed the 25 years just to prepare ourselves for what he really wanted to do.
We've had failed projects since long before LLMs. I think there is a tendency for people to gloss over this (3.) regardless, but working with an LLM it tends to become obvious much more quickly, without investing tens/hundreds of person-hours. I know it's not perfect, but I find a lot of the things people complain about would've been a problem either way - especially when people think they are going to go from 'hello world' to SaaS-billionaire in an hour.
I think mastery of the problem domain is still important, and until we have effectively infinite context windows (that work perfectly), you will need to understand how and when to refactor to maximize quality and relevance of data in context.
well according to xianshou's profile they work in finance so it makes sense to me that they would gloss over the hard part of programming when describing how AI is going to improve it
Working in one domain does not preclude knowledge of others. I work in cybersec but spent my first working decade in construction estimation for institutional builds. I can talk confidently about firewalls or the hospital you want to build.
No need to make assumptions based on a one-line hacker news profile.
Whenever I have to wake for something that I absolutely can’t miss, I set 2-3 extra reminders 5 minutes apart precisely because of this “silent alarm” bug. It’s only happened to me a couple of times but twice was enough to completely destroy my trust in the alarm. The first time I thought I just did something in my sleep to cause it, but the UI shows it as if the alarm worked. I’m lucky to have the privilege that if I oversleep an hour or so it’s no big deal, otherwise ye olde tabletop alarm clock would be back.
Hah - I also just assumed that I was turning the alarm off in my sleep without noticing. I started doubting it and really wish there was a log of when you tapped snooze or stopped the alarm...
This is too much of a dev feature for apple to implement and there are probably third party apps that do this, but meh
