This guy probably never did anything nontrivial with the API - you notice almost... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

moonchrome on Aug 5, 2023 | parent | context | favorite | on: Non-determinism in GPT-4 is caused by Sparse MoE

This guy probably never did anything nontrivial with the API - you notice almost instantly that the chat models (both 3.5 and 4) are nondeterministic at 0 temperature. Source - built a documentation search bot and had it crap out on me on copy pasted prompts when I was demoing it.

cratermoon on Aug 5, 2023 | [–]

Apparently, and I haven't tested this, just from what I read, the simpler GPT-2 models are deterministic at 0 temperature.

keskival on Aug 7, 2023 | [–]

If you want to make it deterministic, just cache the responses keyed by queries.

Consider applying for YC's Fall 2026 batch! Applications are open till July 27.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact