are there any public benchmarks for measuring misalignment?

this is a complete beginner q, how does rl scale given the reward hypothesis (sutton & barto) holds: that an agent's entire purpose is just the maximization of the expected value of the cumulative sum of the reward, a single scalar signal. surely, formulating goals in terms of a single number is extremely limiting. to accept that, no no matter how effective of a reward signal you design, it will always be "your way of communicating to the agent what you want achieved, not how you want it achieved" if this is all you have how could you ever hope to solve alignment or reward hacking?

Today we're releasing FoldingBench, a benchmark for measuring how well generalist foundation models can fold proteins Frontier LLMs still score far below specialist biology models and do not beat a random baseline

This is notable. DeepSeek, a lab usually first to pioneer novel algorithms and architectures, is saying that at this point, the ROI of improving data quality far exceeds that of working on novel post-training algorithms. I think this has already been true for some time for non-lab practitioners. If you're doing llm post-training, 80% of your effort should go into looking at your data. This means: - Hiring experts to dig through your RL tasks - Sifting through rollouts and sft data by hand to remove suspicious samples. Make sure all tasks are actually passable. - Making sure your data is diverse in both difficulty and category.