• 2 Posts
  • 89 Comments
Joined 3 years ago
cake
Cake day: March 22nd, 2024

help-circle



  • It doesn’t literally mean “90% failure rate vs 100%”

    In the HLE benchmark, for example, we may be talking about 45% test score for MiMo Pro or GLM or something, vs 55% for Sonnet and 61% for maxed out Claude.

    But further complicating that, those particular open weight models do better than Claude in one particular hallucination benchmark, meaning Claude/ChatGPT might be worse at RAG/extraction type workloads. It’s like this across the board, with different models trading blows in different areas.

    By “90%,” he really means “it’s so close the better model depends on the particular application, workload, harness, sampling configuration” and such, and that none of these models are close to “100% accuracy” across the board.


    One other thing: Anthropic, OpenAI, and Grok have all been accused of “dumbing down” models under high demand to speed up inference. If your application depends on them, and they are the only provider, that could throw a wrench in the workflow.

    But open weights? Pick your provider. You get an API as consistent as you are willing to pay for.

    In other words, that “100%” isn’t really representing the potential inconsistency one gets being locked to a single provider.













  • Now “Evil Dragons Exterminate Humanity While Humans and Demi-Humans Bicker Over Politics”! There’s a keeper

    Wherever those BioWare employees land, they should do fantasy space opera, like Shadowrun. With that premise. In space.

    Make it Bioware-y, with drama and sexy elves and armor lights and lens flare.

    Both for something fresh, but also to spite EA.



  • brucethemoose@lemmy.worldtoGames@lemmy.world•EA: Ruin Everything
    link
    fedilink
    English
    arrow-up
    1
    ·
    edit-2
    3 days ago

    Look. I’m all for a new world.

    But there’s nothing wrong with building on a good, existing constructed world, too. They can do both. You mentioned publishers that did that quite successfully, at the top of this thread: Owlcat, Paradox, and especially Larian.

    It’s also quite practical from a development perspective. It’s easier to pitch to get funding, they already have lore experts on the team, mechanics they don’t have to invent from scratch.

    I’m just not buying that Mass Effect is a “dead end.” It’s a great setting, there’s tons to explore, and they’ve already broken away from the main trap of being stuck with Sheperd. Halo is a bit different not because they wrote themselves into a corner (which ME aid also guilty of), but that they utterly refused to leave it, amongst other things.


    I will say that I think BioWare just deteriorated.

    …There’s no getting around that, whether they start from scratch or not.

    I have the same opinion of Halo. I vividly remember forming it upon playing Halo 4, with how cheesy it felt.


  • No it’s not. They can use the entire world they constructed, its history, and (for the players) all the nostalgia. And selectively bring back characters.

    I mean, that immediately drew me into this trailer.

    Halo is an entirely different beast. They could not let the MC/Cortana go like they are stuck to them with a chain, but side stories in that world would have been interesting too.


  • Mass Effect is a wonderfully constructed world; I disagree on you there.

    I’d also argue ME3 is not the peak. The best gameplay, sure, but a more questionable campaign IMO.

    You are not wrong about the story trajectory though. They wrote themselves into a corner with the Reaper arc, and that is done now. But I do think “Shepard-free, smaller story spinoffs” were the correct move, rather than trying to pick up a story that should be done (like Halo did).