Hacker News new | past | comments | ask | show | jobs | submit login
Launch HN: Polymath Robotics (YC S22) – General autonomy for industrial vehicles
172 points by stefan8r on Aug 1, 2022 | hide | past | favorite | 73 comments
Hey HN,

I’m Stefan, one of the founders of Polymath Robotics (https://www.polymathrobotics.com) and formerly of Starsky Robotics (YC S16). My co-founder is Ilia Baranov, a career roboticist. We’re building general autonomy software for industrial vehicles.

We’ve just come out of stealth with a full autonomy stack freely available in sim online. Whether it’s a tractor on a farm or a 400t dump truck in a mine, we make it easy for you to make it driverless. Where before you would need an industrial vehicle, $25-150k in sensors + compute, and 6-18 months of building time, now you can start building applications that command autonomous robots right away.

See a demo here: https://www.loom.com/share/852d54744d45452d926b9016f61f5e43.

Two big things are hard about automating industrial vehicles: first, robotics is deceptively hard. But also, industry is nitpicky. These two together are too much for most startups. That is, if you put all your energy into the first—the noble effort of getting the robots to actually work— the second will end up killing you. We learned this the hard way at Starsky, and what we’re currently working on at Polymath is a way out of this dilemma.

At Starsky, we were working on autonomous trucks. We knew that was a hard product to build, but I didn’t really have perspective until I went and hung out with SaaS companies afterward. My SaaS friends use horizontal services at every layer of the stack. Payments? Just plug in Stripe. Messaging? Twilio. Libraries and tools exist for everything. Your thing goes a lot faster when you can just buy the rest of the stack.

Contrast that with Starsky, where we were building our own robotics stack completely from scratch. That meant the autonomy itself (our actual product, and really complicated on its own), but also custom teleop, a drive-by-wire system, hardware abstraction layer, and fleet management system to orchestrate all the trucks via an API. Each of those things could be a company in itself! Imagine trying to debug a stack like that when something’s not working.

That’s sort of just how robotics is today. Like software in the 80s, everyone’s rebuilding the whole stack for each new project. So the tech gets super complex, brittle, and unadaptable. In fact, an open secret in the robotics world is that most demo videos you see are “cooked,” because the robots are so unreliable in the field.

I got to know many of the folks building industrial autonomy vehicles from 2016 onwards, and saw a depressing pattern. Brilliant, mission-driven engineers would get a start with a healthy seed round from investors who thought it was “just” an execution problem. After 18-30 months the teams would have a prototype that somewhat worked, but what they wouldn’t have yet was the commercial traction needed to justify more fundraising.

That brings us to the other half of the dilemma: industrial customers are nitpicky. When it comes to scaling an autonomous vehicle POC (proof of concept) with a large industrial company, they’re surprisingly unimpressed with autonomy. Sure - the first time they see a driverless vehicle their jaws drop and they ask about safety. But like everyone else, they quickly assume it “just works” and start asking whether it will integrate with their Oracle instance. Then they start nitpicking the specific way the tractor drives across the field / the luggage tow stops on the apron / the dump truck pulls up to the processor / etc etc.

In other words, after the initial POC, industrial autonomy stops being sexy-super-hard-technical and starts looking like Enterprise SaaS. Except by then the team has spent 2 years heads down making autonomy (the hard part!) actually work rather than talking to customers. As a result, they’re not up on the top 20 things customers care about and they don’t end up with product market fit. Few startups make it out of that pincer squeeze.

All this to say … what we’re building at Polymath is a shortcut to autonomy for industrial vehicles. We make it so you spend less time building / maintaining undifferentiated basics, and focus instead on the 5-10% of the application that’s hyper specific to your industry / vehicle / customer. Just like our friends in SaaS.

Since we’re just building the autonomy layer, we can focus on getting the tech reliable, stable, and scalable. Our users can build their own custom behaviors, or integrate with apps, or whatever it is that makes their robot actually useful, on top of the Polymath autonomy core.

The software can be installed on virtually any large outdoor vehicle, as long as it operates in a controlled environment. Focusing exclusively on closed environments is partially a cheat code. It lets us assume the vehicle can come to an immediate stop if there’s trouble, or be teleoperated by a human in dicey situations—all the constraints that mean we don’t have to be Waymo or Cruise.

Polymath autonomy includes localization, navigation, controls tuning, obstacle avoidance, and a safety layer. Importantly, we also built a hardware abstraction layer, which lets the whole thing work across different vehicles and sensors. One thing we’re doing differently this time around is buying (rather than building) parts of our solution wherever possible. Again, part of that “make robotics more like software” thing. So we’re using Formant for teleop, Gazebo for sim, and for customers that need hardware installation, we have a partner called Sygnal that does awesome integration and retrofit work. (And, almost obviously, we’re built on top of ROS [Robot Operating System, the standard framework for building robots).

We also have our own demo vehicle, a tractor affectionately nicknamed ‘Farmonacci’, that we use to run unmanned testing daily. You can see a shiny demo video of it here: https://youtu.be/bP0mNG53bVw And now the part I’m most excited about… You can start building on top of Polymath autonomy today, for free, in sim, with a tool we built called Caladan (yes, the name is Dune-inspired).

Caladan is a set of simulated vehicles and environments where you can interact with Polymath’s autonomy via Rest API. That means you can build autonomous vehicle behaviors and applications in your preferred programming language, without having to touch ROS. So, even non-roboticists can build an autonomous vehicle application pretty easily. To my knowledge, we’re the first to build an autonomy product that’s this accessible, and especially something that you can use for free (it’s funny that roboticists, a group least interested in talking to sales people, is super forced to).

We built Caladan by bringing our autonomy code into a standardized Gazebo environment, with a Rest API that allows you to command it in any language (without needing to know anything about ROS).

The cool thing is, you can transfer the same code you write with Caladan to a real vehicle. We’ve tested it out with Farmonacci, and we’ll select people who are hacking on top of our simulated vehicles and give them testing time on Farmonacci as well. Part of the idea for Polymath is to take robotics off of ‘hard mode,’ and help teams build robots that are stable and reliable right from day one.

For now, it’s entirely free to use Caladan. Since each instance is a cloud-GPU, in the next couple of weeks we’ll ask people to pay if they want consistent access (but there will always be a free version). We want to make our money automating your actual robots, and we’ll also charge if we have to do too much custom stuff for you in sim. But, of course, we have startup-friendly pricing if you talk to us.

So please, sign up for Caladan at polymathrobotics.com and mention that you came through HackerNews (we’ll prioritize spinning up your instance if you do). We can’t wait to see what you all build. We’ll be around in the thread and look forward to your comments, ideas, and feedback!




This is really cool. I was on a team for the first and second DARPA Grand Challenge. The teams (like ours) who had to use actuators, pistons, gears oh-my for steering, acceleration, braking were at a huge disadvantage compared to the teams who were given the drive-by-wire access (CMU/Stanford & VW Touareg partnership(s)). It would've been really nice at the time to get the mechanical aspects all worked out and have a nice autonomy layer to use that was ready and tested. I applaud the work you are doing and know it is NOT easy. I also happen to own some construction equipment and can't imagine how hard it would be to interface with a dual pilot joystick configuration (think skids & dozers). It is a completely different way to think about travel, turning, bucket work, tipping weight. Sounds like you all have some really interesting tough problems to work on. Bravo.


Thanks for the kind words!

Ya - I can't imagine how hectic it was for you guys. I remember hearing Josh Switkes talk about hand building radars for pre-2010 autonomy.

Re:booms/buckets/etc: right now we're just focusing on mobility (making these systems drive from A to B), specifically because those systems are deceptively hard. FWIW I think the guys at Built Robotics have done some cool stuff on it, as have groups like Moog and FP Innovations.


Thanks for the kind words, and awesome that you were on a DGC team!

Absolutely, there is a ton of really interesting controls, ML and general software quality work for us to do.


Congratulations on the launch Stefan and Ilia! I'm a PhD student in robotics and autonomy and I've always wondered why something like this already didn't exist, at least for 90% of the 'doable' field robotics tasks. I think you're closer to what Skydio did in the autonomous drones for enterprise space - abstract the autonomy part and just put a little effort into customer-specific requirements. A couple of quick questions:

1. When you say you're built on top of ROS, do you mean the autonomy stack you'd deploy in an actual robot is built on ROS? Are you using ROS or ROS2?

2. What hardware does your current autonomy stack use? For parts of your stack that'd depend on using deep learning based methods (e.g. any image or lidar data), the models that you'd train would have to use a lot of collected and annotated data specific to a particular problem/industry and this would take a non-trivial amount of time, especially since you said the camera/sensor configuration is not fixed can can potentially be decided I the simulator by the user. How do you plan to tackle this?

PS: Any potential internship opportunities at Polymath in the coming few months that I can apply to?


To answer 2. in some more detail: We generally like to have GPS, cameras and lidar. Optionally, we add in IMU, radar and close in sensors (ie ultrasound).

We are approaching the ML problem somewhat differently, we try to give the robot an understanding of navigability of a space. This is done with semantically segmented images, overlayed with depth data where needed.

Once we have this, we flatten it into a 2d costmap of areas where kinematically (ie: ground clearance, terrain handling ability, allowable areas, etc) the vehicle is allowed to go. This is fed to our planner, which in turn generates valid paths for the vehicle to take.

The particular cameras and lidars used are abstracted away in a Hardware Abstraction Layer (HAL) that I've described a bit elsewhere on this page.

Cheers!


This sounds a lot Jaybridge, acquired by Toyota Research. They had things like grain trailers driving next to harvesters until full, then driving to the edge of the field for pickup... also Giant Dump Trucks, operating within open mines. (These were just marketable examples, I believe the internals were quite general, in the context of "adding autonomy to existing heavily specialized vehicles" without rebuilding them...)


Thanks for sharing Jaybridge. I hadn’t come across them but will reach out / try to dig into them.


Thanks so much for the kind words!

1) We are mostly built on ROS (for better and worse) and are starting to migrate some containers to ROS2. Down the road I see an architecture that has some ROS, some ROS2, and maybe some homemade stuff (or maybe even other frameworks).

2) Ilia should be able to give you a better spec overview, but we we’re generally trying to be as stack agnostic as sanely physically possible. Roboticists have really strong preferences when it comes to hardware (often shaped by which components previously ruined demos in their life), and we want to be able to work with whatever you want to work with.

Reach out re:internships! No promises, but now that we have a stack you can build on for free that will definitely affect our decision process!


Hey all, Ilia here.

I just did a quick follow up with a demo of our API driving our real tractor, and simulated tractor, simultaneously.

The API used is the same, and we are sending the same commands at the same time to each vehicle.

Check it out here: https://youtu.be/86pBFbTHPOA


I work in robotics, I understand that every robotics company builds substantially similar core autonomy for different industrial vehicles, so I definitely see the value of what you're creating. However, every company needs to figure out how their autonomy will handle their specific vehicle's dynamics and their specific physical environment. For vehicle dynamics, they have to model things like wheel slippage on rough terrain, articulating joints on multi-axle vehicles, braking distance while hauling a load. For physical environment, they have to figure out how dust, fog, and poor lighting conditions affect their sensing and perception. Do you intend to make custom sim vehicles for each of your customers? Will you simulate each customer's particular camera/lidar/other sensor in their specific environment?


This is a good, well thought out point that I 60% disagree with (respectfully, of course). I think you're absolutely accurate in describing how robotics is today, and you're definitely partially correct about how it will remain, but I think you're biasing towards thinking robotics (as a field) is exceptional to the types of progress that have happened in software.

To unfairly simplify that assertion, it reminds me of a founder who once told me that "robotics will never be as easy as software. It's just harder and will always be so."

I don't know if I think each autonomy company needs to figure out vehicle dynamics any more than I think each company needs to figure out their cloud infrastructure. At a certain scale (and amount of success) you will certainly need smart people thinking about it, but you can get pretty far on a 90% standard AWS instance. Things like wheel slippage are challenging, but their challenging in similar ways across vehicle types (but different from other tasks you need to work on when building that higher level autonomy).

Similarly, if you assume you can safely stop when there's danger, there are a relatively finite number of environmental conditions you should operate in - which we can build, maintain, and improve "once" as opposed to making every mining OEM or Ag autonomy shop build from scratch. In time, there might be certain portions of this stack that you can swap out with your own (or with some other company that's better at seeing in the fog, for example); but I fundamentally don't believe that every robotics company needs to solve all of these hard problems well on their own for every application.

Re:Sim - for paying customers we are putting their specific vehicle (and sometimes their environment) as a part of the offering. We're also using sim to help figure out how to position sensors (which will be another cool thing that we want to show at some point).


I only have experience working in the robotics field as it exists today, so you're right that I am probably unaware of how things are done in the software world. (I mean I read HN so I'm at least somewhat aware, but I don't have first hand experience of it in my work, so maybe I just don't know what I'm missing). So in that sense, I'm rooting for your success in launching this company -- it would help accelerate and expand the robotics field!

I think the way you're bounding your addressable problem space -- industrial vehicles operating in a closed environment -- makes a lot of sense. I also agree that certain classes of problems (like wheel slippage) would be better solved "once" by an expert team rather than repeatedly by people who are actually trying to solve a different problem of higher level autonomy as you put it.

Some random other thoughts:

- your business model and offering sound kinda similar to [Tangram Vision](https://www.tangramvision.com/about-tangram-vision), though not close enough to where you would be competitors. I'm not affiliated with Tangram at all, I've never even interacted with them, just sharing what I've come across in my research.

- How do you think about how your customers join and potentially leave your platform? In your reply you mention a certain scale and amount of success where companies may decide to start doing this themselves, or even going to other companies that specialize more than you do (e.g. Applied Intuition for sensor simulation). It might feel a bit early to think about on launch day, I'm just curious about how you think about your company's value prop for customers at different stages of growth. Are you potentially limiting your market to only niche segments and smaller players? Who is your key customer you're targetting?

I appreciate your response and candor, I'm very interested in seeing your company take off!


Thanks - appreciate that. And thanks for reading my response in the right tone (I was nervous no inflection might do me a disservice).

I’m not familiar with Tangram, but thanks for sharing. I’ll take a look!

There is definitely a world where customers “graduate out” of using Polymath, people do the same with Stripe (I know Uber has been at that decision point in the past). There’s probably a reasonable cap on how much we can charge a customer as a result (more likely pegged to the cost of 10-100 FT roboticists) and in time we’ll have to offer enterprise level SLAs.

So far we’ve seen traction with startups, tech consultants who are servicing big industrial co.s, OEMs, and the large industrial co.s themselves.

My hunch is that as we stop “doing things that don’t scale,” we’ll stop serving the big industrial companies themselves (and leave them to our other customer groups).


How does your stack integrate in with that of Rio Tinto?

They have, after all, been running remote and autonomous 100 tonne dump tracks and kilometre+ long trains for seven years now, in addition to being an umbrella across one of the largest mining fleets across the globe.


Hello, Great question, I haven't seen enough technical details on their system to be able to answer definitely. However, from what I've read of their approach, it's fairly similar to the way we do things.

Perhaps this means there's no reason for Rio Tinto to use us, and that's totally fine. There are 1000s of other, much smaller mining companies that can benefit from the same tech, likely at a cheaper price than what Rio Tinto spent developing their own version.

Also, from our experience, these sort of approaches bake in assumptions on particular vehicle types/characteristics. So even in Rio's case, perhaps they have other smaller models, or other types of equipment they haven't automated. In this case, we'd be happy to integrate with their existing command and control software stack.


Rio has full fleet (bobcat, bulldozers, loaders, trains, underground mining speciality (uphole drillers, etc)) ambitions and they're working their way through them all.

[1] https://www.riotinto.com/en/about/innovation/automation

Some of their talent date back to sheep shearing robots in the early 1980s.

In the same geographic region (W.Australia) there are related automata projects such as self recharging drone clouds about tractors, agri-bots (visual identification + weed spraying), integrated geophysical air survey, etc.

I was somewhat curious as to your awareness of all this (eg: Rio's plans for a full automated "bottom up" $64 billion copper mine in the US (should it go ahead)).

Also, while we're here, do you have any in house mining | industry experience driving tractors, dump trucks, ship loading, etc. yourselves?


I'd challenge this assertion. Rio Tinto has been on the forefront of automation, but there's a long list of equipment they operate that they don't have an immediate pathway to automate.

To my knowledge, the main players automating ultraclass mining trucks are Caterpillar, Komatsu, Hitachi, and ASI. CAT has been working on autonomy for a long time, and to my knowledge only offers full driver-out autonomy on a handful of models. Sandvik also has some really cool see-and-repeat autonomy in underground mining. It doesn't surprise me if Rio Tinto is talking about a full zero entry mine (worth it just because no people to hit = faster driving vehicles = more production/yr), but to my knowledge they buy their autonomy from others and no one I know of (besides us) would automate all the various types of equipment that go into running a mine.

A different large mining co had an initiative to build their own autonomy, but the project got cancelled when it wasn't making fast enough progress and the CEO was replaced with someone more Caterpillar friendly.

We don't have in house mining experience - but again we're not building autonomy just for mining. We're building generalized basic autonomy so the folks starting mining-specific autonomy projects don't get stuck re-building the basic autonomy wheel.


On a technical level, I'm not sure what sheep shearing robots in the 1980s has to do with automating vehicles... but I'd love to see a video/article if you've got one! :D

As with any large opportunity, there are multiple players in the space, both new and old. As seen with the "innovator's dilemma", I'm not sure I'd place my bets on tech straight out of the 1980/90s/2000s (sheep or no sheep ;) )


Congrats on the launch! The novel (for robotics) business structure looks interesting. How are you planning on kickstarting the ecosystem? It looks like you need at least one or two competent companies above you on the stack before you can start getting real-world feedback. (1) Do you have in mind who these companies will be? Do they exist yet or do the need to be created? (2) One of the super difficult things about robotics is reconciling the difference between a clean sim and a messy world. How will you get feedback on how your robots are operating and iterate if there’s an independent company sitting between you and the customer?


Thanks! It would be lovely if posting this one time in HN single handedly kicked off that ecosystem -> so everyone reading should quit their jobs and build robots /s

More seriously - there are a surprisingly large number of groups out there building robots and looking for help. Some groups already have not-built-here syndrome, but our hope is that by being a free thing you can just initially build on people will be "Polymath Native."

Your sim question is especially poignant. Our sales line is that reconciling sim:reality is our problem to solve for you. The more real answer is that we're specifically focused on vehicles in environments that are "relatively clean" - i.e. controlled (even if, in fact, they're a landfill). We're also only really selling to technical users - who are hopefully sophisticated to solve >0% of the problems.


Your post reminded me of the early Apple days. There's parallels there with robotics.


Thanks, we sure hope so!


I think this is fundamentally interesting.

I also appreciated the write up. Rooting for you! I think FAR more startups should be focusing on robotics and basically anything to do with meeting fundamental needs IRL.

You did a great job with the description. Makes me want to get off the sidelines and build robotics on your API. (I have startup hyperactivity lol)

Unrelated: do you have to be a YC company to do a Launch HN?


You should get off the sidelines! Post-Starsky I wanted to run away from HardTech / FrontierTech and just do "easy SaaS stuff."

I was deeply upset to realize I was too broken to do something that "easy." (still, really hard). As a mentor of mine, Boom's Blake Scholl, has said - it's sometimes easier to do "hard" things because it's easier to get the motivation to work on them.

Re:Launch HN - I think so? You can always do a Show HN, though!


Well said!

I'm personally working on something hard in SaaS (automatic summary video after every Zoom/Teams SaaS demo).

I think truly hard things are in a lot of ways easier for truly inventive engineers because it's an execution challenge but you basically know for sure there's a market after.

Also hard things can be decomposed. If you solve one part (as you are doing) that's enough. So you can really niche down within something hard and still change the world


Ya video summary is nothing to sneeze at - I'll take my ML tasks as easy as determining if something is a person or a pixel /s

Completely agree on decomposing hard things.


There's always room for one more simultaneous startup ;)


Honestly wish you all the best here! Truly, it's great that you're trying to simplify the deployment of robotics, because that would be great for everyone.

One thing I have to say looking at your website, is that is has serious "Underpants Gnomes" vibes.

1. Build in sim 2. Test in sim ... 3. Deploy on a real robot

Taking something from simulation and having it work "effortlessly" (as your marketing puts it) is sort of the holy grail of robotics. It's something that has confounded researchers and roboticists for a long time, so I'm a little bit skeptical as to whether your product actually lives up to the promises made in your marketing material. It would help if you showed more demos of what this means in a practical sense. In the video on your website it shows a simulated tractor, and then a tractor moving in an open field with no obstacles. This is pretty much the best case scenario, but how does it actually perform in more realistic scenarios?

Going into your FAQ a little, it seems to suggest the process isn't so seamless:

  We’ve built Polymath with the goal of a seamless transition from sim to reality (other than necessary-but-minimal tuning of controls).
"Necessary-but-minimal" is kind of understating how difficult sensor calibration and controller tuning is. It's all the difference between a simulated world and the real world (and also necessitates the need for controls and perception experts, which your marketing seems to imply are not needed or minimally necessary with your product). I'm curious how your product makes this process easier.

It appears like your value-add is that your product simplifies the whole ROS tech stack to some degree, but to me I would imagine transitioning from sim to reality without much work would imply that you have made real advancements in simulation technology. Is that the case? Or does your product mostly address improvements in the development experience?


Hello,

You are asking about the FAQs here: https://www.polymathrobotics.com/product

We are actually a hardware and real robots company first! Our simulator efforts are for two purposes: 1. Allow more people to try out our autonomy core, and build on top of our API 2. Allow our own developers to run testing and tuning in sim.

To your point, we don't expect tuning to purely happen in sim. However, we did have our senior controls engineer just recently tune up a controller in sim for Caladan, and later deploy practically the same thing to the real vehicle, leading to much smoother steering commands. We'll write about that in detail in the future.

Our work is to ensure that API commands that are sent in sim, behave as close to identically as possible on real vehicles, thereby allowing people who build on top of us to focus on higher level software stack (business logic layer, etc). Hence, users of our API don't need to have any robotics experience, they simply command the vehicle to do something, and we ensure it gets done. (and our team is comprised of perception, controls, ML, etc engineers)

The details of how this is done will be a future writeup, but the summary is that we pass sensors through a Hardware Abstraction Layer (HAL), along with kinematic/dynamic configuration and data about the vehicle. This allows the global and local planners to plan for a more generic vehicle (ex: Ackermann steering vs differential steering), while the HAL ensures that the planners don't generate unfeasible or unsafe commands.

Let me know if the above answers your questions!


When you say this:

  they simply command the vehicle to do something, and we ensure it gets done. (and our team is comprised of perception, controls, ML, etc engineers)
Do you mean you're offering control and perception engineering as a service?


No, sorry, let me be more clear.

Users of our API can send high level commands (ex: go to GPS coordinate X), and our software (on vehicle) will ensure it gets done.

Our software is built by a team of roboticists, including ML, controls, etc. The software we write ensures that the real world vehicle responds almost identically to the simulated one.

There are of course limitations, hence we do a commissioning step where we ensure the work we've done in simulation for a particular vehicle (sensor locations, localization fusion of sensors, controls tuning due to kinematics/dynamics etc) are tested and tuned on the real vehicle. This is done before the real vehicle is put into service, and remotely monitored (collection metrics on performance, status of sensors/actuators, etc).

Cheers!


To add a somewhat easy sentance here - we're our actual product is delivered as SaaS (and it's not at Gmail pricing, it's more enterprise). Were you to become a customer we'd be pretty handholding with you to get the thing actually working.

There is a perception stack and a controls tuning stack within that SaaS that we'd be delivering.


Okay thanks this clears up a lot! The … in the process that I noted in my first post sounds like the handholding you’re talking about.


Ilia gave you the more real (and specific answer) but I just want to call out that I love the Underpants Gnome reference. I used to cite it a lot and was frustrated by how many times I had to explain it.

Sim =/= sim =/= sim. By which I mean - when some people say "build autonomy in sim" they mean lots of different things. Sometimes they mean solve all ML and data collection problems in a simulated world. We don't mean that.

When we say build and test in sim, we really mean that for just the earliest phases. Essentially - if you wanted to build Bear Flag Robotics 2.0 you could take our example app (lightweight Python app to tell a tractor to till a field), meaningfully improve it, show it to farmers to get LOIs, raise money, buy + outfit a tractor, and then put our autonomy on that tractor.

In that end state the behaviors you built out in Sim would still mostly work on the real tractor (same API commands both) and you could end up modifying those behaviors the way you would need to make them work in the real world. We would also be working with you closely to make sure our product is working well, solving new problems, and probably doing things that don't scale to help you be successful.


Had to look it up [0], as I didn't know it.

Side note: I'm always happy to learn a new reference, but most of them are heavily US-centric, and for people like me who didn't grow up in the US, and live or lived in the US in their adult life, sometimes it's a bit too frustrating, because there's just too much of it going around.

The worst of all is when the reference is killed by Italian dubbers/translators (in Italy you can rarely watch TV series or movies in their original language, which is very comfortable, but deeply damages your ability to understand cultural references, which most non-US foreigners would get because they watch the original English version of everything).

Anyway, just to give all of you some new perspective of how these things are perceived by some foreigners.

[0]: https://en.wikipedia.org/wiki/Gnomes_(South_Park)


I always liked this reference as I thought it applied to a lot of ML stuff.

Step 1: Collect Data, do some analysis, train a model or two Step 2: … Step 3: Profit!


Stefan, I met you a few times in SF, and remember you - probably 5 years ago. Really liked how you tried to tackle the transportation problem at Starsky. We had a quite long chat one evening, at a birthday party of an Italian friend. I think we even played bocce together :)

Good luck with your new company! Watched the demo; pretty cool stuff. Feel free to reach out to me if I can be of any help (starting a new VC fund), or even just as a sounding board ($my-hn-username @ gmail, or @simon on twitter).


Thanks so much for the note and the kind words. Would love to grab a coffee or something! My email is first @ polymathrobotics.com, but I'll reach out as well!

Maybe we can play bocce instead of coffee?


Nice work! Are you using Gazebo for simulations for training at the back-end? Or as a user-facing interface to demo whatever automation they want to deploy in a sandbox?


We are using Gazebo for working on controls (getting Caladan ready to launch also led to a number of big improvements in controls) but not for ML training.

The point of Caladan is more for you to be able to build specific autonomy functionality without needing a vehicle, or a large plot of land, or a team to build a basic autonomy stack that works.


And, I should say, thanks for the kind words!


Looks interesting! Are you able to make any guarantees for determinism when replaying recorded data through your stack?


Hey, Ilia here :)

Not sure if the question is about the simulator Caladan, or about our autonomy core.

For Caladan, we don't support playback of data currently. Do you have a use-case need for this?

For the autonomy core, the localization, costmap generation, and path planner are all probabilistic to a more or less degree, so we can't guarantee determinism to any highly accurate degree. However, if the same data is played back over and over, the vehicle would react in basically the same way (perhaps not navigating to the exact same spot, but within a small tolerance). Same question here; what is your need/use case to have guaranteed determinism? Is it a safety question? (We can have a much longer chat there)


Thanks, I was talking more about the autonomy core side of things.

So to narrow it down a bit more, my question is a bit tied to safety, but also to general development. Given a set of logged inputs from a machine that had a field issue X, am I able to reliably reproduce (hopefully deterministically) what happened on that machine? (and therefore what went wrong)


Ah ya, debugging in real environments is a very difficult question, particularly in robotics where the system is complex, and not always with the best internet connectivity.

The approach we are taking to this is two fold: 1) Robust self monitoring and metrics, ensuring we measuring system performance (we are using Prometheus to allow us to scale to (hopefully) millions of devices. 2) Ensuring we tie those metrics to a rolling buffer of the last 90 seconds (or so) of data. When an issue occurs, we automatically save to disk, and allow remote operators to pull the raw sensor data as needed.

With the raw data, we can (and do!) play this back through our stack and step through the behavior tree and control outputs, comparing them to what actually happened on device. The system is certainly deterministic enough to allow this sort of debugging and testing!

We will have a set of articles on this approach in the future!


Congratulations Stefan, "Kaggle for robotics" seems like an exciting direction! I still remember your kindness and just wanted to wish you the best.


Thanks so much! Glad to see you’re doing well!


Have you explored using a formal verification tool to prove out the safety/liveness properties of your (or the end user's) autonomy specifications? Could you model the system in something like Spin/NuSMV/PRISM and then ensure that certain properties hold (useful states are reachable and dangerous ones are not)?


Hey,

No, we haven't looked into formal tools like that, good suggestion! TBD on if these approaches bode well for real world robotics, (due to model complexity and model/real world fit problems) but at least there seems to be some applications to path planning and controls: https://ieeexplore.ieee.org/document/5509686

I'll pass it on to the controls team!


Do I understand this right..:

- You have a simulator, like idk The Sims world

- You train a machine in this world, totally virtual

- You deploy the trained machine on real hardware in the physical world

- The machine works good enough that it can perform the task until it can't anymore and then someone remotes-in to fix it?

Did I get this right or can anyone ELI5 to me? I mean it sounds really cool, in theory...


Hey!

We don't train in simulation (if you are talking about ML training). Our ML training is done purely using real world data.

The simulator is to allow developers to "test drive" our autonomy core and API for free.

We have real vehicles as well, but you can imagine it's more expensive, time consuming, and generally slower to let people test on real hardware. Hence, if people like what they see in sim, they can get in touch with us to deploy their code to real machines on a case-by-case basis.


To add to what Ilia said - re: - "The machine works good enough that it can perform the task until it can't anymore and then someone remotes-in to fix it?"

The machine will work good enough for basic autonomy, but then the real application-specific work begins. Whether that work is you modifying the behaviors you're having us do, you routing those systems to humans to help out, or you complaining that our stuff sucks (and us trying to rapidly improve it).


Do you see the platform then providing for now "reptile" autonomy (mobility, navigation, localisation) to get started fast, so customers can focus on their "neo-cortex" applicative needs?

(please let me emphasize this "reptile" autonomy is today perhaps the hardest/time-consuming part killing many startups, as you nicely explain in the OP---Moravec's paradox)


That's a great way to think about it, you are describing it exactly correctly.


> and focus instead on the 5-10% of the application that’s hyper specific to your industry / vehicle / customer. Just like our friends in SaaS.

Ah so the thing that actually takes 90-95% of the time? "Twillo" is no good to me if I dont integrate it properly.

I struggle to see what you've built here other than gym[0] with a largely useless API (Come on, ML training doesn't suck because of lack of HTTP API's), sure its hyper focused on automating vehicle but thats something that's existed for a while - atleast for drones [1].

Shoving sensors on random devices doesn't work that easy, you know that - I dont need a PHD to tell you that. Farmers likely aren't building prod grade ML data sets (except those SWE who gave up FAANG after making $BANK)

What's the real value prop? Why shouldn't I remake the environment in Gym and just cut you out? Gym doesn't require reams of code for environments, same as you.

[0] https://www.gymlibrary.ml [1] https://microsoft.github.io/AirSim/ (2017)


Might be a bit of a misunderstanding here. We’re not providing a sim world for you to do ML training on. Caladan would actually be a pretty terrible tool for that - it’s a static low-res environment without any other agents.

In Caladan we’re giving you a whole autonomous robot (in sim) that you can order around via a simple API.

In our use case the 5-10% that I’m talking about is really hard, but most teams / projects run out of funding before they get to that point (because making the robot, and making it autonomous, is so time and cost intensive it dominates the project).


To double click on the Twilio API example:

In most robotics applications teams are building the equivalent of new speculative phone networks, building specialized infrastructure, acquiring specialized hardware companies to ease computer:phone connections, and running out of money before they can find product market fit for mass texting some sort of consumer.

Sure, they can still integrate Twilio poorly and fail. Or they can build a use case that no one cares about - but at least they won’t need 5 years to get to that point.


"Shoving sensors on random devices doesn't work that easy, you know that - I dont need a PHD to tell you that. "

Definitely, a lot of our work goes into ensuring our Hardware Abstraction Layer (HAL) works well across vehicle types and sensor types. Check out some of my other replies here to double click on this.

For your other notes, Stefan's already covered it, but the TLDR is that we are building on vehicle autonomy, not simulation environments or systems (Caladan is just to get people an easy way to use the Polymath API, without needing a robot first :) )


Hi Stefan, this looks great!

I'm curious to hear what you think your competitive advantage is against other players in this space such as Applied Intuition. From my research, Applied Intuition has a focus in the AV space but has been moving into other verticals. Cheers.


I could be wrong, but I don't think Applied Intuition and Polymath have the same product or business plan. Applied Intuition has a high-fidelity simulator as the main product, Polymath has actual autonomy stack (hardware and software for real world robot) as the product with a low fidelity simulator that gives devs a playground before actually deploying it on a real robot. You can problem train your ML algorithms using the synthetic data from Applied Intuition but Polymath simulator doesn't serve that purpose - they are using real-world data to develop their autonomy stack.


This is exactly correct, Caladan from Polymath Robotics is just a cheap and easy way to get a bunch of developers to try out our (fairly basic so far) API, and to start thinking about the business logic and applications that could be built on top of this.

Our actual product is autonomy on real vehicles.

We don't plan to ever build any kind of high fidelity sim, just sticking to basic Gazebo or similar.


Spot on. Except we’re just the onboard software (BYO HW)


Curious what's your thoughts on using telebotics where human remote operator is in the loop where automation is not possible?


Absolutely part of the plan, there will always be strange situations that the vehicle can't adapt to. So it has to stop and phone home.

Our current test vehicle uses remote operation a lot!


In fact, during our interview with TechCrunch we let Kirsten drive our unmanned test vehicle from her browser at home!


Hi Stefan, thank you for sharing.

I would love you honest take on ROS as an operative system platform for robot development


Congrats on the launch, Stefan. That is a heck of a nice wrap on that tractor. Best of luck!


Thanks so much! We had a fantastic designer, and our marketing/ops/cust success/everything else lead is really way better than we deserve!


You mentioned large vehicles - is there any support for small, e.g., use in a swarm?


Hey, Ilia here :)

This could be built, but unsure it'd make sense in terms of required compute on each small robot. What would be the use case you are interested in?


To add to this - a neat thing about swarm robots is splitting the compute load between robots and figuring out how different robots move in conjunction with eachchoter. We're just focusing on the software that sits on the robot itself - so any of that coordination would be above our layer


I can imagine several autonomous trucks sharing large segments of roads, and splitting into a number of locations at the ends. Such vehicles could benefit from coordination on the shared segmets.

I suppose that trucks serving most open mines work that way: they have to share the common spiral part.


Ish. To _probably_ annoy you and everyone on this thread, I'll tell you how this works for a "large publicly traded company that has autonomous vehicles in mines."

Essentially they have a portion of the mine site designated as the "autonomous pit," which is the only place where autonomous vehicles can operate. Not all vehicles in the autonomous pit are autonomous (basically no one can automate the vast majority of industrial vehicles, which is part of what we want to solve). In order to be able to see these non-autonomous vehicles, this PubCo makes the mine buy a $20-50k transceiver to put on every non-autonomous vehicles (in addition to the $1m auton hw upgrade and $250k/yr in autonomy SaaS).

These autonomous vehicles do have sensing, but they more rely on those transceivers to tell where vehicles they shouldn't hit are.

So ya, sure, it kindof works like how you think it should. But in a worse way.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: