Our company recently adopted an external AI... and it's incredibly expensive.
The personal version resets every week. But the enterprise version doesn't have that haha
It's almost no different from the cost of using the API;
So we've also placed quite a lot of usage restrictions on it.
We're serving an open model for internal use... but it's incredibly slow, and it gets even slower when users gather.
At least there's no token limit, so there are people who use it simply.
And now they're saying they'll put a token limit on the internal LLM too....lololol
I think it would be fine if we just filter out a few heavy users, but those heavy users
will submit an approval request saying 'We need to do this so please lift the token limit'
Of course it'll be approved. Then the internal LLM serving will still be slow.
If they just fully unlocked a high-performance LLM... I'd want to try various things like agent development, you know.
But reality is a mess lol (agent development is blocked. Because they haven't opened up the LLM endpoint...)
