> "the charts below compare it with GPT-5.5, GPT-5.6-Sol, Claude-Opus-4.8, and Claude-Opus-5"
There should be a rule against releasing seemingly-obsolete comparisons without explaining why you're doing it. If you don't give a cost reason, it makes you look like you're trying to hide something.
It is a useful comparison since it more precisely defines the model capabilities. The alternative would be 'quite a ways from Astra' which is less helpful. So I don't think they hide anything, they don't assume people don't know about newer models or won't notice.
I would suspect the point of the older comparisons is to be able to show that local models have caught up to where frontier models were a few months ago.
But this was published yesterday, so at the time of publication it was not (without qualifying context). I'm not saying that the comparison is bad. I'm saying that the comparison only makes sense if you factor in price, which the post doesn't do.
I know this is besides the point but I genuinely remember a time not very long ago that people used to say that they would kill for an claude opus 5 model at home or an open weights model and that opus 4.8/5 are the first models which finally get what they are suggesting.
Seems like our expectations and the floor has been rising at such an exponential rate.
Has someone created a benchmark for our expectations to see how much they've grown for new model releases? I'd be curious to see some of the data :-D
> a benchmark for our expectations to see how much they've grown for new model releases?
Posters have been saying they'd be happy with open-weight Fable 5 since before F5 was public and I saw one poster on HN say it last week. And with those posts there's usually been a few mentioning the truism that goalposts tend to move and we haven't met our future selves or seen the future context we'll be in.
Studies show humans are poor predictors of our own future wants, needs, preferences and behaviors and few are fully aware how far and often we gradually move goalposts.
I know that but given that it's most likely seems to be an open-weights model or that you can still run theoretically at home without being at the whims of Anthropic/OpenAI is still quite extraordinary.
Seemingly has links to GitHub and HuggingFace, but no actually explanation about what we're supposed to be able to find there "soon". Is this supposed to be downloadable weights? Open weights? Who knows?
One link to "https://longcat.ai/chat/" is provided, which eventually redirects you to https://passport.mykeeta.com/pc/login, did the UI get hacked or something? I went to mykeeta.com which doesn't respond, passport.mykeeta.com serves OpenResty's default page, not sure what's going on but looks strange.
Asking "Hello?" in the chat you get "你好！我是 LongCat，美团研发的大模型助手。有什么可以帮你的吗？" back, which is an explanation what LongCat is in Mandarin. If you then write "What?", it does web-searches trying to explain what "What?" means, in Mandarin.
I'm not really sure this is ready to even be played around with yet. Some more basic prompts seems to tell me that despite the benchmark results, it's fairly dumb, so gut tells me this is benchmaxxed then pushed out as quickly as possible.
There should be a rule against releasing seemingly-obsolete comparisons without explaining why you're doing it. If you don't give a cost reason, it makes you look like you're trying to hide something.
Seems like our expectations and the floor has been rising at such an exponential rate.
Has someone created a benchmark for our expectations to see how much they've grown for new model releases? I'd be curious to see some of the data :-D
Posters have been saying they'd be happy with open-weight Fable 5 since before F5 was public and I saw one poster on HN say it last week. And with those posts there's usually been a few mentioning the truism that goalposts tend to move and we haven't met our future selves or seen the future context we'll be in.
Studies show humans are poor predictors of our own future wants, needs, preferences and behaviors and few are fully aware how far and often we gradually move goalposts.
I don't think you're going to be running this 1.6 trillion parameter model at home.
One link to "https://longcat.ai/chat/" is provided, which eventually redirects you to https://passport.mykeeta.com/pc/login, did the UI get hacked or something? I went to mykeeta.com which doesn't respond, passport.mykeeta.com serves OpenResty's default page, not sure what's going on but looks strange.
Asking "Hello?" in the chat you get "你好！我是 LongCat，美团研发的大模型助手。有什么可以帮你的吗？" back, which is an explanation what LongCat is in Mandarin. If you then write "What?", it does web-searches trying to explain what "What?" means, in Mandarin.
I'm not really sure this is ready to even be played around with yet. Some more basic prompts seems to tell me that despite the benchmark results, it's fairly dumb, so gut tells me this is benchmaxxed then pushed out as quickly as possible.
On benchmarks they seem comparable. The wild cards are the cache hit rate and speed.