Evaluation of Robotics is fully broken today. Repeated calibration, manual set ups, lack of standardized frameworks or scoring methodologies. All of these are time sinks that make it 100x slower than LLMs. Longer cycles, lesser iterations, lesser progress. WE MUST FIX THIS

03.10.26 Lake Tahoe

non-grid portable energy infra wins with current demand nuclear prob wins smth like @RadiantNuclear

Game theory lens on how incumbent tech companies with monopolies should integrate personal AI agents — if you don’t integrate you risk losing dominance — if you do integrate you risk losing revenue ($80bn) @mvernal suggests a premium tier for seamless agentic shopping exp.

Love having more evals my friends work on

Congratulations @chargu_ and the @FireworksAI_HQ team on releasing Specialized Intelligence Index Domain-specialized Evals are the future of how we define what good looks like for frontier models!

autonomous robots is a prerequisite for successful space colonization

4/ Raise your standards so high people think you're a psychopath. Screw being average. 5/ Never follow the conventional path. It's full of boredom and has the most competition.

Standard simulated environments lack real-world physics and noise. Interactive World Models get closer, but training them needs video labelled with low-level actions, and real footage rarely has those labels. Today, we’re releasing the Reka Inverse Dynamics Model (RIDM), a compact model that extracts low-level camera/motor commands directly from raw video. 🕹️Trained entirely on video games 🌐Generalizes to real-world footage 🔐 Weights open-sourced under Apache 2.0 Read the research note and grab the weights on 🤗Hugging Face: reka.ai/news/reka-inverse-dy… #MachineLearning #Robotics #WorldModels #AIResearch #OpenSource

Interesting concept of generative video to extend perception

If you are a - engineer with 2+ yrs of exp - based in SF - fascinated with robotics, infrastructure and energy - FDE type beat Exciting space co. might be ur next work home. DMs open 👀

Good evals make me happy :)

Congratulations @arthvid and the @withdavidai team for releasing DAI-ASR-I18N Shows where current frontier audio models really stand. Multilingual diarization and transcription. The benchmark audio models need!

This v cool!

Introducing MC-EgoHands, the first all-in-one harness for turning raw ego video into training-ready data. → Frontier hand pose (<7mm) across 4 public benchmarks → Human grasps retargeted to Panda/LIBERO and Yam in one pass → Occluded frames flagged as interpolated or missing → Structured episodes in @huggingface @LeRobotHF v3

I definitely manifested this. 1 day later @PantheonInc releases Argus: SOTA open source annotator model for granular episode annotations @MidcenturyAI releases MC-EgoHands: recovers hand poses and makes it trainable action data These are the companies turning raw data to intelligent signals to unlock robot learning!

Amazing work by @MidcenturyAI to go from ego video to hand poses for training I can attest that this was a massive undertaking even 8 months ago, fine tuning WiLoR or HaMeR but now MC EgoHands seems to have solved it well! Great day for robust annotations!

Failure data is crucial for learning Categorizing Failure Data from episodes of open source datasets might be one the highest signal annotations you can get for you data… now it’s finally open source Great initiative by @PantheonInc @ericli_ 🚀🚀

One of the strongest example of benchmarking annotation agreement and alignment across the strongest models today. Data annotation QC and agreement are downstream of a great data annot pipeline

An open source robotics data annotation pipeline is game changer for the robotics data and learning community!

Argus: an open-source robotics data annotation and quality pipeline. With new frontier VLMs like GPT-6 Astra, we can generate rich, high-quality annotations for robotics data for both training and dataset analysis. Argus delivers detailed, timestamp-level annotations while catching issues like mislabeled instructions, sped-up recordings, swapped camera streams, and unflagged operator mistakes. To support better data quality for the robotics community, we’re open-sourcing Argus for anyone to use 🧵