• Scipitie@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    4
    ·
    7 hours ago

    Not sure I’d trust a benchmark where haiku beats opus in anything but cost or cost/performance at least.

    Even though a more open model would be nice to see up there this seems … weird.

    • lindgrenj6@lemmy.sdf.org
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 hour ago

      It’s funny because it seems like this specific test proves more about refusals than actual performance.