speaker-0: Hi everyone and welcome to the Robotics Stack Podcast episode number nine. My name is Leo. Today we are first going to deep dive into OneX, the Palo Alto-based humanoid company. And we are going to extensively cover the story and most importantly their new breakthrough hands. After that, we are going to look into robot surgery.

speaker-1: My name is Oscar. So today

speaker-0: Maybe you've heard of it. It's more common than in the US than it is in Europe, for example. Then we cover construction robots. And finally, we are looking at benchmarks and how do you evaluate basically robot training and robot policy.

speaker-1: Let's dive right into 1x technologies. I mean, they have been in the news for a couple of different reasons, but ⁓ mostly their breakthrough hands recently, which I'll get to in just a moment. But the company itself is also quite remarkable. They are also not that old, but also not that young. You know, we had this in the past, many companies in the robotic space are two, three years old. But they were founded in 2014, and initially they were called HelloDo Robotics. And if you think that sounds a bit weird, yes, it's because they are from Norway originally, and their CEO is Bernd Owend Bornig. And they have actually scaled quite a bit to 200 plus employees right now. Moved to Palo Alto, California, of course. And they also have a factory in Hayward. Now they raised around 130 million, so they had a Series B with around 100 million led by EQT Ventures. In 2024. They get funded by the OpenAI Startup Fund as well, Tiger Global, Samsung Next, and a couple of others. There were also reports that they were seeking up to one billion at a ten billion valuation just last year. I think we discussed it as well in private. But so far this hasn't been announced, this hasn't been completed, but they're really making big steps. So what is their robot? They have a humanoid, which is their flagship robot. It's called Neo, it's around Five foot six, and they also call it on their website the home robot. So they design it specifically, and you see that in the design as well, for the home, unstructured environments, and of course the coexistence with humans. So that's really their branding. The robot is actually already available since late 2025 for $20,000. So it's a medium price point, I would say. And alternatively, what is interesting, you can also do a subscription. For 499 per month, you can rent the robot basically. And the first 10,000 units have been sold out already. So quite quick, but the pre-order is still open. So if you wanna buy one of such robots, you can go on the website and order it, pre-order it. Now, in terms of scaling, they really will not stop with these 10,000 units. So they plan to scale to 100,000 units next year. So another company that joins this race of massively scaling up their robot production. Which is insane. And ⁓ therefore they're also planning another facility in San Carlos. They produce a lot of things in-house: the motors, the sensors, and of course their breakthrough hands. If you think about the design, because I think their design is quite unique. They have sort of a knit suit. so they really think about the texture, how the robot feels, you know, not just like plain robotics metal, like we see it with the T eight hundred, for example. And it also has these like button. Like eyes. It gives me a sort of a Baymax vibe, kind of friendly, but also a bit unusual, I would say. Because that's when I first saw this robot, it kind of felt a bit unusual to me. But now as I was researching the company more, I went to the website and I actually saw quite a bit of high-res pictures, close-ups of the robot face. And I kinda changed my mind a little bit after seeing these. Because yeah, a couple of people criticized the robot for their design. But yeah, maybe it's worth actually seeing it in reality and then judging the robot, which is always the best thing. They also think about the sound. ⁓ you know, if you have a uni tree or if you've ever seen a uni tree on a video where there's no music, they're not dancing, it's actually quite loud and the Neo is actually less loud than a modern refrigerator. So it's pretty calm and blends in in your natural environment at home. I just like their approach of thinking about the psychological implications of robots as well. And if you go to their website, you can see they're really, really big on design. They have an amazing website. Definitely check it out. But of course, the big question is what about the utility? How useful is their robot, right? And what do they want you to use it for? Their idea is essentially that you give the robot a list of chores. So you schedule time and then you come back later to a cleaner, you know, more organized home. You know, I'm big about cleaning. I think cleaning and organizing is one of the main use cases for humanoid robots at home if we're talking direct to customer business. So I think this is a good approach. But still, the robot comes with something called expert mode. So while it does work autonomously by default, there are chores obviously it doesn't know. And for that you have to schedule an expert meeting with a one X expert to guide it, essentially teleoperate the robot in your home. Okay. So This was obviously also criticized by many people because apparently there's not that much that it can do out of the box. I mean, we are both familiar with the models, we know the capabilities, but for the average person, you know, you might look at the robot, you order it, and then maybe they think, okay, it can do everything out of the box, right? Which is ⁓ just not the case. But for them specifically, for operation, they have two different systems. So one is called Redwood AI, and this is like their core policy slash controller. It's a vision language action model. And then recently, a couple months ago, they announced 1XWM. WM stands for world model. So they also have their own world model. And this one is integrated into the NEO as a robot policy. So they mix VIA and Word model. Now what can it really do autonomously so far? Obviously, it can basically navigate. So it can walk, it can climb stairs, it can sit down or kneel, it can have a conversation with you. ⁓ obviously there's an onboard AI, you can speak to it, you can give it voice commands. it can do simple fetching, for example, fetching a bottle of water or different objects. Some reports said 80% success. Of course, that's not production ready. You know, 80% success is pretty good, but it's it's not, hey, give me this water bottle, had eighty percent success, it will give you the water bottle. So that's not how it works. You need one hundred percent for most cases. Opening doors, I've seen ninety percent success on that one. ⁓ light tidying, ⁓ wiping surfaces, putting away dishes and cups, also self-charging, basic environmental awareness seems to work. And of course the hope is that with the new warp model features they can generalize some unseen tasks straight up from the video stream. So in one of their articles that they have on the page, they even speak about one-shot learning. So well, I I'm I'm rooting for them. I hope they they're gonna achieve that. Now I know a lot of people have criticized them and said, like, okay, this is teleoperation, this is really not ⁓ true autonomous behavior. But in my perception, I think they're actually doing a smart move because they're deploying the robot right at your home, so right where you want it. Then they teleoperate it, which we know is also a way to capture data and also train the model potentially. And then you can pretty much use the environment that the robot works in. Anyway, so you don't need to have a super general model necessarily. Maybe you can even have a specific one or like a almost fine-tuned version for this specific customer after three months of continuously operating through teleoperation in your home, for example. To me, this approach really makes sense, ⁓ given the fact that we are not there yet with full autonomy anyway. So obviously for many of these tasks we have discussed it, the hands are a crucial aspect. And on July 9th, so just a couple weeks ago, they showed their hands off. And these hands, I mean, you've probably seen them as well. They look absolutely amazing, right? They they look like alien technology basically. They have these super nice shots of the hand where you see like the tendons and all this kind of stuff. It's really a breakthrough one. I'm curious to hear your thoughts, Leo, in just a moment, because you are also a hand guy. So ⁓ let's let's look a bit at at the key specs. They kind of frame it as the API to the physical world. You know, always gotta have a beautiful catchphrase there. Actually these hands ship on production neos. So it's not like something separate, it's not something you can buy separately. These come with a robot. And I don't think they plan to to ⁓ separate them. And they have twenty-five degrees of freedom in total, twenty-two ⁓ fully actuated fingers and the palm, three at the wrist. They're tenon driven, so they have motors in the Four arm and these motors run on a proprietary low ratio. So five to one gear, ⁓ to fifteen to one gear. So quasi direct drive. And in the industry, a lot of the times you would see a hundred to one gear or even two hundred to one gear. So this obviously gives a lot more control about what you're actually doing with the hand. The joints read write. And this is quite interesting because also on the article on the website they talk about okay, how does the human actually use his hand? Well, it's almost like a tool of discovery, right? It goes somewhere, it discovers okay, this surface might be a little slippery, this is metal, and then it kinda adjusts the movement on the way. So that's why these hands are natively force controlled and fully backdrivable. So you push a finger while precisely reporting the force and the proprioception. So proprioception, if you're familiar with this concept, it's basically if you close your eyes as a human. ⁓ you have a feeling where your joints are in the space, even if you don't see them, right? And that's what you what you wanna achieve with these robot hands as well, specifically. Because it enables very gentle manipulation and ⁓ rich perception of action, not just vision. They have a tactile skin for very high resolution sensing of force, contact location and shear across the fingertips. They have real-time slip detection and adjustment. They're also quite strong, so the peak torque for these is 3.5 newton meters for the thumb, 2.6 for the fingers, and for the wrist it's 17 ⁓ five. And the distal flexion is 45 newton. So it can pick up quite heavy things as well. It's also quite durable. It has a IP68 waterproof rating and it's food safe, so the robot can essentially go and wash the hands himself. And if you look at the videos, it's it's remarkable that they achieved that because it's such high-end and also sometimes fragile technology with these hands because of all these parts. They're working together almost like a clockwork. They already tested it millions of cycles. It has very low distal inertia and well, it's starting production. So hundreds are already built in their in-house production line. They build it end to end completely. Okay. also quite remarkable and they're targeting 10,000 units this year. What can it do so far? They have shared some videos assembling Lego, for example, pick individual screws and coins, plug USB C, spin install light bulbs, use a screwdriver, zip a jacket. You can watch all these examples on the website. It's quite quite ⁓ entertaining. And if you just look at them, like they have so much flexibility And they even show them being used for sign language and stuff like that. So imagine like talking to a robot in sign language. This is just ⁓ end level future stuff. And of course, everything that they do with these hands, they they capture the data, they have labeled data, they can use this data for learning. I think the CEO himself says these cross the threshold for daily human hand tests, and he says the hardware ceiling is gone. I hope he's right. It's a very bold statement, but ⁓ if you look at these hands, I mean it's not unrealistic.

speaker-0: Curious about the last example you mentioned about sign languages, because let's say if you have to communicate with someone in sign language and you have no knowledge of sign language, you pretty much cannot communicate. So that that could be a great ⁓ tool to have like a humanoid robot as a translator, because you can just talk to the robots, the robots will pretty much understand what you tell him, and then it can basically talk in sign language using its hands. And at the same time, it can also get the sign language input from the other person through its vision sensors and translate it back into spoken or written language to you. Right. It would not be as easy to do with a smartphone, for example, ⁓ or maybe with I don't know, smart glasses or something. So it's like one thing a robot could be like super well designed to to do.

speaker-1: Yeah, it just looks absolutely badass as well. You know, like ⁓ this this kind of mechanic movements. It just blew me away when I saw it, to be honest.

speaker-0: Overall on OneX, ⁓ I mean, I've seen the company around like everyone else. it seems to me that their CEO is really, really a strong humanoid hobbyist. That he has been in the space for a while. He really loves what he does. You know he's not like an opportunistic entrepreneur just in it for the money. ⁓ he's been there before everybody else. I mean, may maybe not before Boston Dynamics, but nobody was there before Boston Dynamics anyway. So yeah, I think he has a pretty good edge. And like you said, the data collection and goal is pretty strong. We've seen many companies ⁓ trying out ⁓ like ⁓ live benchmarks like Unitry, opening contests for task completions, which are data collection ⁓ events in disguise. We also see companies like Altronics setting up a robot park where there is like dozens of workspaces for teleoperation. for data acquisition, but we still don't have real world deployment for data acquisition besides what JD is doing in China. And since it's in China, it's obviously a bit difficult to get accurate information about what's actually going on. So I think it's a pretty smart move to go for data acquisitions through teleoperations that way. But it still raises questions like teleoperation means someone will take control in the world inside your home. Probably someone in India, Bangladesh or Vietnam. If you are aware of that, do you still want it? And at the same time ⁓ I mean I hope they are not overly marketing their products. ⁓ I'm not sure the robot will be as capable as what they are out of the box. So I'm just curious is that at what point should you say it's mostly teleoperation for data acquisition and in the future it will be autonomous versus it's already fully autonomous and you know, not actually delivering on the premises.

speaker-1: Yeah, I mean that remains to be seen, but with this hand they definitely got me on board. And also about the teleoperation in your home, obviously there are privacy concerns with this. But at the same time, if you get someone to clean your house, right, you need to trust them as well. And they come in person. So what is what is the less invading thing to you? Obviously it's a big company, they're getting the data from your house. And we've seen a couple of companies with motion capture and ⁓ egocentric data capture devices offering you to clean their house for free. We've discussed it in a in a part before. So for me personally, I would say, hey, give me this robot. And if after a couple of months he does it fully autonomously, I'm a happy client, right?

speaker-0: Yes. Yeah of course. One thing you could argue between like having someone to clean your home versus having a robot is that the person who comes to clean your home will stay for like one hour and then leave, you know, and they do that once a week. While for the robot, the robot will stay in your home pretty much forever. To me, I still feel a bit uneasy with this. It's like having a camera in your home. You don't know where you know if there is someone watching or not, but eventually I will have to get used to it because it's definitely coming.

speaker-1: Yeah. I mean with those many data breaches, ⁓ at some point I'm just like, take all my data. Of course it's very important, but I think you can implement systems that for example anonymize things. You know, I'm thinking about let's say you leave a couple of documents on the table, you know, and the robot walks around and he records these documents and maybe there's confidential information on there. So you could have some post processing of the video, for example, where you blur such things automatically. These are the things that I think about. So

speaker-0: I guess that you are completely sold on having a humanoid robot full time in your home, right?

speaker-1: Yeah, yeah I am.

speaker-0: Th then I have an another question for you. ⁓ would you get medical attention or even surgery from a robot?

speaker-1: I mean, that's a good layup for the next topic definitely. And I think yes, I mean it depends on the very specific case, of course, and what has been the experience in the past. Am I the first one to receive this robot ⁓ surgery or am I like patient number two thousand? Okay, then I don't really care, then anyone can do

speaker-0: Yeah, so you are not that much of a first adopter in the end, I guess. But then if

speaker-1: I mean, I'm definitely not an early adopter for surgery, for sure. I would wait at least a couple of hundred people before me, please.

speaker-0: Well, maybe one day the one ex-robot gets like a surgery skill or something. Who knows? But yeah. Yeah, why why not? Why not? You know, if it's an emergency or something. I don't know. Yeah, true. Yeah. But the thing is, surgery robots have already been around for a while, actually. ⁓ so they are called robots, but they don't look anything like humanoids. ⁓ I mean, maybe we should show a picture of what they look like.

speaker-1: Heart surgery at home.

speaker-0: But you could imagine some kind of large half overreaching arm that has then two to five sub arms holding surgery tools at the tip of it. One of the main ⁓ manufacturers of such robots is American. It's called Intuitive Surgery and their flagship product is called Sudavinci. One interesting thing is that this is a company created in ⁓ the early two thousand and ⁓ right now They have thousands of these surgery robots around. Mostly in the US, I haven't seen that many ⁓ cases in Europe as well, maybe in high-end hospitals. But one thing that is for sure is that in order to use this robot, which is not autonomous at all, which is why I prefer to call it a tool, is that the surgeon will control it 100% end-to-end. And what they allow for is that you still need a trained certified surgeon to ⁓ manipulate it, right, to operate it. And it allows for faster and safer procedure because they are much more precise than using, you know, ⁓ their bare or gloved hands with traditional tools. They also require, for example, in the surgery we are going to discuss, it's called a laparoscopy, which is a removal of the gallbladder. And in this case, for example, if you were to perform the surgery manually, you would have to basically open the patient on a wider angle which is not convenient. So robot assisted surgery in this case allows for safer and basically faster surgeries. The main problem in a way, such a machine on average costs two million upfront and recurring costs are, you know, around 200,000 per year in maintenance, additional tools and so on. And a surgeon who wants to use it takes approximately two months of training on it. The way it works is that so there is the the large robot head that is on the patient side that I mentioned. But there is also some kind of operating booth that the surgeon uses. So it's like some kind of early VR headset where instead of rearing the headset, you put your head in some kind of say some kind of a booth to be completely emerged into it. And then the surgeon has like two different ⁓ hand pads to manipulate it, two different controllers, one on each end. And they also also have foot pedals. Control the robots. So now we get to the point of what I'm going to talk about. ⁓ a research team ⁓ led by Zeki Liang from UC San Diego released on July 8th an article in Nature, which is maybe the most widely recognized scientific publication in the world. And it's also not common at all for nature to publish things about AI, robotics, or you know, virtual reality new technologies. And so what they did in this project. Is that instead of using the Da Vinci surgery robots, they used two humanoids, two Unitary G1. So again, this is not autonomous surgery, this is controlled surgery uses using the robots as tools, right? And the robots were operating on live bigs, not on humans, obviously. ⁓ But it is still to demonstrate that it is possible to perform pretty much the same thing with a much less expensive ⁓ you know tooling. Because again, a DaVinci soldiery robot is approximately 2 million dollars. And a United G1 in this case is approximately ⁓ 14,000. So the cost difference is pretty impressive. They were using the same booth I mentioned on the surgeon side to control the robots. Because what happened is that UC San Diego had a donation of an older soldier robot controller. And this controller is compatible with ROS. which is robot operating system. So they were able to programmatically bridge the gap between the controller and the unitary robots. So the unitary robots in this case were holding the typical surgical instrument, you know, that a surgeon would use. And they made two different demonstrations. One of them was with two unitary robots and another one was with one unitary robot and a human assistant. Just like a surgery where the surgeon would be assisted by human. And it did work. In both cases and they perform laparoscopic surgeries on live pigs and the pigs survived after that. So it is still pretty impressive. This is a very impressive demonstration of feasibility to me. The one caveat is that it is a bit less precise on typical tests. So for example, when you train as a surgeon, I didn't know that something I just learned, there is something called an O-ring transfer test, which is you have A little plate with vertical pegs on the table and with ⁓ I don't know what they call like tweezers, you are supposed to take some little rings and place them from one peg to another. And the the surgeons who were performing the surgery they trained to control the unitary robots doing this test before that. Because since it's something they do as part of surgery training, it kind of ⁓ you know gave them a good feel of how the robots ⁓ answer to them. And this was approximately Two times slower, and the actual surgery was approximately ⁓ five times slower. But at the same time, ⁓ you have to keep in mind that it is five times slower than using the actual Da Vinci surgery robots. But if you were to perform the surgery manually, it would be closer to nine times slower, right? So in every case, the surgery performed ⁓ these two haki humanoid unitary robots are pretty much ranking between. using the dedicated Da Vinci surgery robots and a human doing it manually. When I say human, I meant a surgeon, obviously not a random human. So they mentioned in the paper as well something that is in my opinion a little bit misleading is that they say okay the surgeon they only needed you know the five minute training doing the ⁓ o-ring on peg exercise to handle the robots. ⁓ but what you understand reading the entire paper and that they forgot to mention is that these surgeons were already trained on the Da Vinci. you know, machine. They already went through the two months approximately process and they have experience using this machine in their professional life. So they kind of know how to use the control booth I mentioned already. So for them, it takes them five minutes to go from the Da Vinci to the unitary. It doesn't take them five minutes to go from nothing to the unitary, right? So I think it's important to underline. So it also means that a surgeon who is already trained on this can without further warning or training ⁓ pretty much use the Unitree instead of you know the Da Vinci, which is still pretty good. But a surgeon with no experience on robot surgery would not be able to, you know, ⁓ use it. One thing to keep in mind about that is hardware autonomy, if we can picture it this way. It means that it's pretty clear how the booth controls the surgery robots. And changing the hardware on the patient side is pretty much visible and is kind of a what's say w a step further towards open source surgery hardware, right? Even though DaVinci is not open source and UT is not open source. But meaning that you can kind of hack a control boost to make it use any output hardware is still a pretty, you know, ⁓ significant story in my opinion. So yeah, I I I like the fact that basically ⁓ we are going hardware agnostic on, you know, surgery robots. ⁓ intuitive has a pretty large market share with their Da Vinci ⁓ surgery robots. So I think it's nice to have more cost effective alternatives in a way, because not all hospitals can, you know, afford a two million machine.

speaker-1: Yeah, so I mean we have this very important ⁓ phrasing in the AI space and that is this is as bad as it gets. If this applies to the surgery, I'm all for it. I mean surgery is getting faster, or maybe not directly faster as you pointed out, but also scalable because you know, even producing these high end Da Vinci takes probably longer than maybe a Uni tree robot, which as we've seen, they scale up almost towards infinity right now. And the more you have, the more people you can obviously operate. Of course you also need to scale the surgeons, but ⁓ maybe there's gonna be a surgeon robot as well soon.

speaker-0: Yeah, definitely. Maybe there is no case to have a two million machine to perform surgery anymore, you know. ⁓ maybe we can divide the cost by ten. And it's also I think a strong win for teleoperation. And I know you like teleoperation and it's just one research team that did that, not like a two thousand people company with a ten million dollar budget. So I think it is a pretty strong case for both hardware and teleoperation.

speaker-1: I mean yes I do like teleoperation, but I also like the different touch points of robotics. So we have the humanoids at home, of course, we have the surgery ones, possibly in the hospital. But what about construction? Because at the end of the day, you know, there's so many machines out there in the world. And especially in construction, why not automate them? Why not make them fully robotic? And this is ⁓ what we get to with this topic, because one company, which is called Terra Firma. Just got funded and they are in the area of construction robotics and tech-enabled earthworks. I want to use this to to talk about them a bit and also the competition because they're not the only one in the space. There are quite a few. And it seems a bit under the radar. So TerraFirma, they got funded with 115 million recently. Very young company, founded in 2023, and they are headquartered in Austin. ⁓ Texas or Buda. It's not clear. I think they relocated or expanded to Buddha, also because they have now testing grounds there. And 100 million of this funding that they received comes from a Series A led by Kleiner Perkins with participation of Bain Capital Ventures, which is an earlier backer of them, and a couple others. Also Angels from SpaceX and Undereal. Because the team is of two X SpaceX people. What they focus on is human in the loop, semi-autonomous robotics for Earth moving and site preparation. So we talk about excavation, grading, things like that. Making constructions faster, cheaper, safer, you know, all these keywords that you mentioned with surgery as well. And you know, the long term version, obviously they are both coming from SpaceX, is to extend this to Mars infrastructure actually. So terraforming and you know regular handling. Because the founders, they definitely want to make sure that you know that they came from SpaceX, I I bet. And that's why they named their company Terra Firma, right? Because you also have the Terra Fab of ⁓ SpaceX. But anyway, they're ⁓ two interesting guys and they actually landed their first deal with a landlord they had after building robots with Tupperware. At least that's what they just recently said in an interview. ⁓ the CEO is Noah Shotchit. He led design and mass production of Starlink user terminals and also Starship Rockets at SpaceX. And he has a engineering background from Princeton. And the second co-founder has the same first name, also Noah, but Noah McGuinness. He's the CTO and he developed flight software for SpaceX satellite constellation. So Starlink and Star Shield, and he also comes from Princeton. This is actually where they met. So They're kind of trying to apply the SpaceX rapid iteration approach, first principles thinking, you know, to the construction bottlenecks. They also have some key leadership, ⁓ including the head of operations, Angelo Kirchon, he's from Energy X, and ⁓ also others with backgrounds at Tesla or Underreal. The company size is obviously still quite small because they are so new, but what is exactly their product, you might ask? Okay. What is exactly the service that they're offering? So as I said, human and the loop. We we're not talking about full autonomy yet. You know, it's not fully unsupervised or something like that. But human and the loop, planning, orchestration, and operation for these construction sites. They have three major tools. So the first one is the mission planner, and this is sort of their AI-enabled pre-construction software for bidding, planning, simulation, and execution. So everything you do before you go to a construction site, basically. Then you have the mission control. Which is sort of their system during the process. And this is a remote command and control center for coordinating operations. And the third is kind of the hardware part. They retrofit heavy machinery. So excavators, doses, loaders, rollers, skid steers, etc., for remote operation by small crews. So they make it so that these big machines actually support autonomous workflows. And they also implement multi-layer safety. For example, they implement person or obstacle detection because. You know, construction is also quite a dangerous field. That's why they all walk around with helmets, of course, right? And if you're semi-autonomously operating there, it might become even more dangerous. So you wanna put an emphasis on safety here. And this leads to ⁓ one operator ideally controlling multiple machines at the same time. So three X productivity gains here. Their business model is that they're basically a contractor slash service provider. So they perform earthworks. projects themselves. Right now they're focusing on Texas. ⁓ but of course planning to expand. So they have real-world projects in Texas. For example, a Starbucks site excavation in North Austin. Then the Zachary Cole Foundation Retreat Center. Then they have also partnerships with the commercial players and the US government for infrastructure and logistics. And they're growing their fleet. I'm not sure if they actually own all of these things themselves, but I would assume so. At the same time, similar to the hands, we can also use this to generate data or to capture data basically. And this data can be used to improve their models. Because actually, and this is kind of what surprised me when I was researching them, this whole area of automation on ⁓ construction sites is I would say it's exploding kind of. You have a lot of different companies in there. For example, you have also Build Robotics. Some people have heard of it. They also do these retrofit kits. for excavators and doses. And these retrofitting kits usually have cameras, of course, you know, maybe they have lighter, they have all sorts of sensors, IMUs. So yeah, you know, we know it all already from robotics. So pretty much they they they put the robotics knowledge onto existing machinery. Built robotics, for example, they are even older. They were ⁓ founded in 2016 and they have already 40 deployments and they focus on solar. For example, solar and wind energy. So all these little companies usually have their own kind of focus area. Then we have bedrock robotics. They also do retrofit kits for heavy equipment. They actually have a X Waymo team. So we see a lot of overlap between autonomous driving as well, you know, car industry, robotics. So it makes sense that they have the Waymo team on there or the X Waymo team. They strongly focus on their AI stack, the operator software for 24-7 work. And they had some really, really big raises. So they actually raised 350 million total. So that's that's bedrock robotics. Then we have AIM, intelligent machines. They also do the retrofitting. And their focus area is, for example, mining. So you can also go into mining. Or they also have a US Air Force contract. And then besides them, and these are just the startups, then besides them you obviously also have the original equipment manufacturers because you would think, okay, ⁓ what about the the machinery companies themselves? Don't they want to automate their stuff as well? Why do they need these startups? Well of course they also exist. So for example the famous Caterpillar and they have the cat command, the MindStar software, hardware stack basically for remote operation. Then we have Komatsu and they do smart construction. So they also have advanced autonomous systems. They are very big in Japan and global earth moving. And then we even have Volvo CE or Hitashi or John Deere. And these are also offering machine control, remote operations, and increasing autonomy features. So this is a really, really big market, of course, you know, and automating this will probably make a lot of people rich. Because imagine in in 10 years, 20 years from now, you want to build a house and then suddenly all these autonomous vehicles come on. Come on, you know, they excavate everything. Then robots jump in, they start laying the fundamentals of your house or pulling the pipes and everything. And then you can basically create houses almost fully autonomously. And I think that's how that's re yeah.

speaker-0: House as a service kind of product.

speaker-1: Yeah, house as a service. It's called rent, by the way. Yeah. No, but that's pretty much pretty much about them.

speaker-0: Right. I see. Yeah, I think construction is definitely ⁓ like a strong case for robotics and automation, ⁓ because of labor shortage, obviously. And it's also a difficult job that fewer and fewer people want to do. But ⁓ it also has is technical difficulties, like ⁓ it requires strength, most of the work you have to do is unstable, ⁓ it's pretty dangerous. So yeah, I think it's a pretty strong case for robots as well and probably more worthwhile than robotic surgery in a way when it comes to autonomy.

speaker-1: The the question for me is just is this the right approach? Because I remember back when ⁓ Tesla and ⁓ car autonomy was a big thing. I mean it's still an ongoing thing obviously. George Hotts, ⁓ I don't know if you know him, he started this company called Comma AI and it's basically also a retrofit kit that you can use for your car and then your car that originally has not the capabilities for autonomous driving can suddenly drive autonomously.

speaker-0: Yeah, I've seen it at the time. I don't I don't know what they are up to right now.

speaker-1: Yeah, that's the question. I mean, I think they're still releasing stuff and they have some audience, but the question to me is just if you you know, it's like own every part of the stack and you probably have the biggest advantage. And I think that's what these major companies like Caterpillar obviously think about as well.

speaker-0: Yeah, similarly, I mean it's going a bit off topic, but I've seen a startup recently that makes some kind of device that you can add to your mechanical watch or any watch to turn it into a smartwatch. And I think it's interesting because not everybody likes to have an Apple Watch, but I I'm not sure either. Is it really something people will actually want? Does it make sense to retrofit? ⁓ does it make sense to turn like a non-digital device, an analog device pretty much into a digital device? And once again, the longest case, but not the easiest one, is for humanoids. If you replace humanoids can, I don't know, dig whatever lay pipes, like you said, it can also drive the machine if you if you have the right model and the right training. Otherwise humanoids would be the best, like ⁓ I would say the best case for

speaker-1: this. Yeah, that's right. But but how do we actually how do we actually make sure that humanoids keep improving? You know, I've heard benchmarks are playing a huge role here.

speaker-0: Yeah, yeah, and it's actually the the next topic here because benchmarks in robotics are super hard and also super broken at the moment. So it's important we discuss them. So you probably know about benchmarks for LLMs. I mean they are all over the place. Every time Enthropic OpenAI or whatever release a new model, they say, ⁓ we score ninety nine percent on whatever benchmark which is better than our competitors model and so on and it's a never ending, you know, positive feedback loop for us users in the end. But you have to keep in mind that it's pretty easy actually to pick benchmarks for LLMs because they are 100% software based. So you just have to plug your LLM into the Eval system and you get a result, right? And right now I think benchmarks for LLMs are pretty good. There were early versions like a couple years ago for example where you could overoptimize your model to be high performing on the benchmark, but not that good at real world practice, for example. And I think right now that gap has become pretty narrow. So an LLM that scores better on a benchmark is also actually a better tool for you to use on a daily basis. But robotics has not reached that point yet, mostly because of the seam to real issue. that we mentioned. So a sim to real problem is basically an unsolved problem in robotics, and I'm not sure it will ever be 100% solved. It will always remain an open topic. It's basically how do you go from a model that performs well in simulation to the same level of performance in the real world. And there is no real answer to that yet. ⁓ world models are kind of a good intuition, which is if we have a model that understands physics, then we can pretty much translate from simulation To the real world ⁓ seamlessly. So basically what happens right now is that most benchmarks in robotics are based on simulation. ⁓ the main the main benchmark is called Libero, and it has 130 tasks that are based on language. So it's what we called a language-conditioned manipulation task. So there is a written text instructions and the robot is expected to complete the task. There is also Calvin. ⁓ which is long horizon tasks, ⁓ which is a task that requires the model and the robots to ⁓ break in down into several steps in order to ⁓ complete it. Like a kitchen recipe, for example, you know, it's like you don't make it in one go, you need to divide into steps. ⁓ Robocassa, which is kitchen task, and there are also a bunch of others like MetaWorld, RL Bench, and so on. Basically anyone can take on these benchmarks, these challenges and have their score you know openly displayed on their leaderboards. But the issue is that over the past 16 months, average success went from 75% to 98%. Right? So you would think 98% on those benchmarks will mean I would already have a robot in my home, you know, doing my cleaning and my dishes, like we talked about ⁓ earlier. But it's not the case. And why is that? Well it's because if you take A real world challenge like the 2025 behavior challenge, the result, the success rate across 50 domestic tasks was actually 26%. So this is a pretty strong gap. 91% on benchmark and 26% in the real world ⁓ is pretty significant. So some people are trying to fix that. ⁓ in October 20-2025, a Chinese company called DexMal, ⁓ in collaboration with Hugging Face, they launched robot challenge. So it's still an online real robot revolution with a benchmark and they also have presence at research conference. ⁓ so Dexman is a Chinese startup and they make basically robot foundational models and Hugging Face maintains the dataset. What's novel about them is that they score in the real world and they score both on success rate which is the binary and they also have a progress score that rates overall partial completion or you know good direction when it comes to completing the task. And ⁓ the thing is all of these measurements are itself unstable and it also probably has to do with the conditions in which you test the environment and also the type of robots that perform it because all robots are not the same. You know, even from one to another so don't necessarily have the same joints, have the same ⁓ embodiment and so on. So Yeah, it is still pretty difficult. There is no standard, no one size fits all, and also we don't really know ⁓ what to say what would make sense long term to evaluate for robots. For example, robot challenge, ⁓ they mostly evaluate two arms with claim kind of tasks. But the same task could also very much be, you know, performed by a humanoid, right? So it is still difficult to say, for example, if you are entering this challenge and your goal is to score very high. Would you rather use a humanoids that has more degree of freedoms, but is much more complex to get to work, or you know, just two robots around this claims that are easier to set up. And also one of the issues is that in June 2026, a team of researchers actually asked themselves the question: what are we actually benchmarking in robot manipulation? And they tested the Libero, the benchmark I mentioned previously, and they said that only approximately 20% of the Libero completed tasks were actually significant. Because what they did is that they took a model that had no language understanding at all, right? But the model was still trained on, you know, the task that TBRO is evaluating upon. So they still managed to score pretty high because the model was pretty much overfitting the task from the change, right? So basically the model was pretty much ignoring the instruction of the task. But it knew how to complete the task because it was trained on them. So it means, you know, for example, if a task is like grab the red cube and place it in the box. And just imagine you have a bunch of colored cubes. If the instruction changes and it says, okay, grab the blue cube instead of the red one, ⁓ the model will completely fail. You know? So it's a typical case of overfitting. And in this case, inner resistruction will still very much work for the model. And it does it does remind me that Hugging face was, you know, a strong contributor in LLM benchmarks at the beginning, for example. They were making tasks without providing the instructions in order for the model developers to not be able to overfit the task they were looking into. So I think we are definitely ⁓ going into that as well for robots benchmark. And ⁓ I've been wondering about it a lot. Is for example, I've been looking at dexterity quite a lot lately. And ⁓ there are many different approach to train and evaluate and also have precise, dexterous manipulations with hands and fingers, right? And ⁓ so I'm wondering if there is space for a benchmark just for hands or a benchmark for single hands with one hands or for I don't know, size manipulations with you know ⁓ the robot hands. And I think in this case ⁓ maybe there is an edge here to dive into, right? ⁓ something very precise, which is not necessarily long horizon. It can be real world tasks, maybe not as complex as cleaning the house or something, but you could still have the hands to, for example, ⁓ peel an apple, for example, I've seen that. Or, you know, manipulate an egg without cracking it and so on. So I think when it comes to yeah, benchmark and It can also be a good way to perform data acquisition. I think we would have we will see definitely more and more specific benchmark for more and more specific tasks and that there will not be, unlike for LLM, a one size fits all benchmark or model that can how to say ⁓ cover everything, right? Because I think robotics is first much die much more diverse and also the real world. challenge like the confrontation to physical reality is what makes it harder to regeneralize overall.

speaker-1: Yeah. There was someone on X and she definitely made the point that benchmarks cause the industry to optimize for them to some extent, obviously. Yes. And that and that obviously will also cause innovation with that in mind. So yeah, maybe benchmarks are the way we have seen it with ARC AGI and these benchmarks, and they are I mean there's there's always almost like a little overlap with robotics and LM benchmarks because obviously you can also use LMs in the part of the vision language model, vision language action model, etcetera. So to some degree these benchmarks of traditional AI have some overlap there.

speaker-0: Yeah, exactly. But still the physical reality makes it more complex because you would have to gather all the robots you are evaluating in the same place at the same time in order to reproduce all the tasks in one sitting. So yeah, it's still a very much an open question and I think whoever solves it has a pretty strong business or a pretty strong contribution to the field anyway. So yeah. That's all for today, I'm guessing. So it was very insightful episode. ⁓ Still very information dense in my opinion. So yeah, it was very nice to have you here. Please consider subscribing to our channel if you enjoyed it. We do weekly podcasts on everything new and interesting in robotics. Also, we are pretty active on Twitter. And yeah, this was the Robotics Tax episode 9. My name is Leo.

speaker-1: My name is Oscar. See ya. See you.