Daniel Abreu Marques: Welcome to Autonomy Insiders, the show where global industry leaders in autonomous driving unpack their real-world insights.

Aryldo RUSSO: the idea is that we need to have different points of view to be able to approve the whole system. it's also a kind of heritage from the guided transportation systems. But in the case of an autonomous transportation system, there is seven different domains. the manufacturer do not control the deployed risk. So the manufacturer is responsible for the vehicle itself and that it's capable to drive But they cannot define everything and limits that the system that will be in place. will operate.

Daniel Abreu Marques: Welcome back to Autonomy Insiders. I'm your host, Daniel. And getting an AV type approved in Europe is one problem. And getting a driverless service actually running on a specific street with a control room behind it and a city that has to agree is a completely different one. And France built a legal framework for their second one. And most of the industry outside France has never heard of it. So today we're going to take it apart. And my guest is Aryldo Russo, Group Innovation Director at GESTE Engineering. Aryldo is a chartered engineer and fellow of the Institution of Railway Signal Engineers with more than 25 years working on safety critical transport, control command, ramps, software assurance, and independent assessment across Europe and internationally. And GESTE is an independent Swiss engineering consultancy founded in 1999 with teams in Switzerland, France, and Belgium. And in France, GESTE is approved as an OQA STRA, so a qualified body for the safety assessment of automated road transport systems, included the overall system safety. So Aryldo welcome to Autonomy Insiders.

Aryldo RUSSO: Thank you very much for the invitation

Daniel Abreu Marques: So before we get to your work, I think we need to set the scene for people because there is an institution at the center of the story that almost nobody outside France has heard of. So the body that oversees automated road transport in France is called the STRMTG. That is the only time I'm going to say those letters after this, I will call it the French technical service. So maybe tell us what the French Technical Service is, where it comes from and how it ended up responsible for driverless vehicles and explain what that origin says also about how France decided to treat this problem.

Aryldo RUSSO: Yeah, so in fact, STRMTG it's the national technical service in France, but it was at the beginning responsible for the guided transportation system. like metro, tramways and something like that. Different from the high-speed railways that is treated by different one, but STRMTG is for the city transportation. And so based on that, the moment that France decides to introduce autonomous vehicles, this was pretty much the idea to use what was already applied in the guided transportation and try to do not a copy paste, but use the basics on the autonomous vehicles as well. So the main point is that in the guided transportation systems, the system is treated as a whole, not only the vehicle, but all the system, all the interfaces and so on. So this is the idea that they have in mind to apply as well in the autonomous systems.

Daniel Abreu Marques: And GESTE's origin is also in rail and guided transport. So when you look at an automated road transport project today, what do you see that a team coming maybe purely from automotive does not see?

Aryldo RUSSO: Yeah, so we have a more than 25 years experience in guided transportation. So you try to put the same experience in autonomous vehicles. The main difference is this one that I've just talking about is that people from the automotive engineering thinking a lot about the vehicle itself. And the principle is not just to have the vehicle approved, but all the environment infrastructure, communication and services that goes together. So this is what we try to use when we evaluate or when we help people to develop the demonstration. And I think that this is the key point when you talk about the system as a whole.

Daniel Abreu Marques: And maybe let's get concrete about your role because most listeners have no picture of it. So can you walk us through what actually happens when when a project calls you? So a shuttle operator or a driving system developer picks up the phone. So what is the first meeting and also what do you ask them for?

Aryldo RUSSO: Yeah, so the idea is that what we want to see it's what's the intended services. So who is organizer? What is the operator? What kind of vehicles will be put in place? What kind of systems proposed? And moreover, all the routes or zones or to be a passenger or freight, the speeds that they have in mind. And basically the environment that the system will be integrated on. After that, of course, we need to check all the dependencies that they have. is the ODD that is the vehicle the system is already approved to use or it's capable to use in these kinds of projects. At the end, we ask as well if they have already prepared any kind of basic safety demonstration like hazard analysis or if the vehicle is already type approved if there is any kind of scenario and test strategy that has to be put in place, taking into consideration the whole system like cybersecurity operation, maintenance, configuration, and so on. So all this kind of information are used to build the gap between what has been already approved and what has to be approved in order to compose a whole system.

Daniel Abreu Marques: And your approval as an independent assessor is also granted domain by domain, so not as a single license, and you hold most of them, including the overall safety assessment of the complete system. So can you name the domains for us and also maybe why does France slice the approval that way?

Aryldo RUSSO: Yeah, in fact, it's not that the approval is sliced in this way, but the idea is that we need to have different points of view to be able to approve the whole system. it's also a kind of heritage from the guided transportation systems. But in the case of an autonomous transportation system, there is seven different domains. So basically they are reliability and safety onboard systems. reliability and safety of connectivity and positioning systems, cybersecurity, safety of the infrastructure or the road infrastructure and equipment, safety of the vehicle behavior. This is pretty much the application of SOTIF. We have operation and safety management system that is what is being put in place after the deployment. And of course, the domain number seven is the overall system that collects the information of the six previous domains and builds a conclusion about if the system is okay to operate or not. But it's pretty much that we have several point of views that has to be collected and consolidated in just one report. What I mean is that there is no such of a certification of Just one or two domains at the end, we have to cover the overall system.

Daniel Abreu Marques: Makes sense. And also the rules require the assessor to be independent of the designer, the operator and the service organizer, but you also offer engineering support, for example, to the applicants. and in a market is this small with very few qualified bodies. How do you keep those two roles apart in practice? And maybe also has the separations ever cost you a project?

Aryldo RUSSO: Yeah, this is true. to be able to work as an OQA, you have to be independent of the whole development process. So what we do basically is that first we have independent teams that the people that are going to work on the evaluation has never been working before on the conceptual phase, the development phase. And we have to evaluate project by project and having a kind of background evidences to be sure that the people that will be working on the evaluation is not involved or it was not involved at least in the last five years in the project that we are evaluating. So, but in any case, it's a kind of trade off that we have to do all the time. Sometimes it's better to go to the support instead of certification. It depends on the project by project. And as a... the advancement of the project is not yet there to be fully certified for the moment. We have some projects that we work on the support side and projects that we are working on the certification side.

Daniel Abreu Marques: maybe now let's get into the framework itself. So I want to build this from the ground up because I suspect ninety percent of the people listening have have not heard of it. So maybe let's start with the definition. So in French law, what counts as an automated road transport system and and maybe also just as important, what does not?

Aryldo RUSSO: Yeah, so basically what is written in the decretes and in old guides is that we have several layers of systems and so on. So we do have the technical systems that is what we mean about, we can pretty much talking about the vehicle plus ADS. And then we have the full system that includes infrastructure, connectivity, supervision and for the development part. And then when we are going to deploy, there is also the operation and maintenance. the idea is that the guided public transportation cover, the guided transportation system decree is also pretty much the same that covers in the SCRA, so autonomous vehicles. transportation systems.

Daniel Abreu Marques: And one thing I find structurally interesting is that France did not build this through the vehicle law, so it built it through the transport code. And the starting point was also criminal liability. So why that route? What does it tell you about what French regulators were actually worried about?

Aryldo RUSSO: Yeah, it's not really the criminal side, but it's pretty much the liability on. We are going to, we are talking about public transportation system. So this has to be clear. So we're not talk about a particular robotaxis for example. We are talking about a mini buses or buses or something like that, that is managed by some high level operator. We need to know that who is in charge or who is liable for each part of the system to be able to, in the case of an accident, how we are going to work with that. So it's much more than just a kind of criminal point of view, but it's to define who is responsible for each part of the system. This is if we have a problem, an accident or any other kind of problem, we have a clear view of which one we have to treat with that.

Daniel Abreu Marques: And the framework also creates legal actors that do not exist in EU type approval. So the remote intervener, the service organizer. So why does France put the obligations on the organization running the service rather than on the manufacturer who built the vehicle?

Aryldo RUSSO: Yeah, the pretty much the point on that is that the manufacturer do not control the deployed risk. So the manufacturer is responsible for the vehicle itself and that it's capable to drive in a straight avenue or something like that. But they cannot define everything and limits that the system that will be in place. will operate. So the operator control, the daily operation, maintenance, staff, and remote intervention, all this border information that is not really related to the manufacturer. So based on that, we are able to say, some risks are covered by the vehicle and ADS. Some risks are covered by procedures. Some risks are covered by the other information other than just an in-rent safety of the vehicle. Pretty much this is the big view.

Daniel Abreu Marques: it leads also to a question about the ceiling of the of this whole thing. So the scope is also predefined routes or zones with remote supervision. So is that a deliberate limit or a transitional one? And also let's put it that way. So if if Waymo showed up in Paris tomorrow and they want a free floating robotaxi fleet across the entire city, does this framework have a shape for that?

Aryldo RUSSO: This is something that we need to evaluate in a different way. means, prodefine the route does not mean just to connect the point A to point B. We can talk about zones, for example. There is some experimental projects that are already doing this kind of thing in a small cities, for example. So the idea is that we can evaluate and approve or not all, but the main possible routes in a city. And based on that, we can have an authorization to cover the city as a whole. So the point is not just that we are going to go to A from B using just the same route every time, but the idea is that we are able to evaluate that this vehicle is capable to take any possible route in the city from the point A to B without any other constraints. this is the idea. Of course, as bigger the city is, much more difficult is to evaluate. But for the moment, this is what is recognized in France.

Daniel Abreu Marques: And now let's get to the concept at the heart of it all. So France also requires a new system to be in plain English globally, at least as safe as what it replaces. So explain that to somebody hearing it for the first time. So at least as safe as what exactly is it then the human driver or is it the bus service that runs there today?

Aryldo RUSSO: Yeah, in fact, this is also a concept that came from the rail transportations. And before it was called Gamab and then it was transformed to GAME So GAME is globally at least equivalent, but it's not related to equipment itself, but it's pretty much for the safety of the transportation systems. So what it means is if we have a kind of bus transportation systems with driver. The idea is that if we put an autonomous bus, roughly the number of accidents that we are going to have, will be at maximum the same that we had in previous with a normal transportation systems that we are replacing off. So this is the concept of a GAME So we cannot be more dangerous that's what has been already in place.

Daniel Abreu Marques: And is it then the average basically? So for example, if you look at you want to replace one specific route with an autonomous bus, is it then the statistics on this specific route or is it the global statistics?

Aryldo RUSSO: It's the global one, pretty much like normal bus transportation systems has a safety level or a level of accidents of X. So the idea is that if we replace this for another one, it has to be at least as safe than the one that existed before.

Daniel Abreu Marques: And who actually decides at the end? So you write an opinion, but somebody also needs to sign it. So can you walk us through who holds a pen and also what happens when your opinion and their decision maybe point in different directions?

Aryldo RUSSO: Yeah, that's true. In fact, who decides to put the system in operation is the operator at the end. What we do is we prepare a report saying that in our point of view, the system is ready to be operated. It's safely ready to be operated. then based on that and based on several other different evidences and tests and whatever the operator has in hand. It's the operator that decides to put the system in operation and then requests to the mayor to give him the authorization based on all the evidence that exists. So it's pretty much, and the mayor in this case is the mayor of the village. It's a little bit different from the US and so on. It's much lower level.

Daniel Abreu Marques: maybe let's also get one level deeper into the mechanics. So there are three dossiers in one sequence. So first the technical system design dossier, then the preliminary safety dossier, and then the final safety dossier before the system enters the service. So what decisions get made at each stage and at which of the three do projects most often stall?

Aryldo RUSSO: Yeah, there is several other dossiers that has to be prepared in parallel as support dossiers, but pretty much the sequence is that. So the first one that we have is the DCST. So it's the one that if we make a parallel with the rail transportation system as well, is the one that defines the generic system. pretty much what we have in our facility that is ready to use. It's a generic one. we have a generic ODD that it's capable to cover and so on The characteristics of infrastructure that they are capable to cope, communication that's necessary, the supervision that's necessary in a generic point of view. So this is the first approval that we have. The second one is the DPS is the preliminary safety case, we can say. that is the deployment of this generic system in any specific one. So we have something there that can be used whatever we want. So we are going to evaluate in this specific route or this specific village or whatever, how it copes. And the first thing that we are going to find it, there is some gaps. For example, I was driving in Spain two weeks ago and I found out that The speed limit can be the normal one, the round one with the numbers, but there in Spain there is also a square one blue with the numbers that's also related to the speed limit. So this is something that can happen also when we are going to deploy an autonomous system. So it's capable to read some types of signaling, but there's others that is specific to that village that they have to be configured to work on that. So this is what is inside of the DPS. And the DPS is the one that proves that we are capable to go to the test phase. So pretty much it describes the system deployed in a specific route or zone and list all the tests that we have to perform to guarantee that the system will be safe at the end. Finally, we have the DSO, it's the final safety case that will collect all the results of the demonstration, the tests and whatever, and see if everything is covered and if the system is really ready to operate from the safety point of view. So this is the last one and based on these three documents, we can prepare a final report and deliver the final report to the operator.

Daniel Abreu Marques: And when should a developer first talk to an assesor and maybe also be honest about the flip side. So what does it cost the project when they come too late to you? So with the engineering already finished, for example?

Aryldo RUSSO: Yeah, so as soon as possible normally. So what we mean, the evaluation of the systems begins at beginning of the project. The risk to begin later is that the evaluator will find something that has to be changed. So we have to come back to the beginning of the project and change whatever is necessary to cope with the requirements. to be able to develop something that is safe. So basically, the evaluation begins at the beginning of the project. On the other hand, it will not impose more time to reach the end because as it will follow the development process, as soon as the questions are raised, the answer will come and so it follows the normal development cycle. So this is really not very a problem. And also for the price, I cannot give you the amount of money for a project, specific project, normally this is not what costs a lot. it does not cost a lot if it follows the normal development cycle. Of course, it comes at the end. we decide that we have to change everything because there is nothing that copes with the requirements, it will have an overcost that is not negligible.

Daniel Abreu Marques: Sure. And maybe let's also talk about remote supervision. So there's probably no settled answer in the industry on how many vehicles one person can supervise, on takeover times or on fatigue. So what does the French framework actually require? And also, yeah, what do you see the teams consistently getting wrong about that?

Aryldo RUSSO: Yeah, there is no numbers that is defined in the law. What is defined that doesn't matter the number of vehicles that we are going to supervise, we have to have strong evidence that is what is possible to be done. we are always looking at the demonstration or the evidence that we have to support some hypothesis instead of just the number. Because the number can be completely different. For example, a normal weather with no traffic and so on, maybe someone can supervise 10 vehicles. But if there is a bad weather with a big traffic in the football game aside and so on and so on, maybe we can just supervise one or two. So all the evidences and justification has to come along to be able to evaluate and say, okay, this is something that is feasible. We agree with that. And in a certain safety level, it's acceptable or not.

Daniel Abreu Marques: And does the French law also require the remote operators to sit in France or can they sit where wherever they want more or less?

Aryldo RUSSO: For the moment, we are talking about be sitting in France and pretty much nearby the operations side. There is nothing explicit in the law, but for the moment it's been easy or at least all the experimental projects that we have to be as close as possible from the operations side. But I think that is just because they are experimental projects, so it's pretty easy to do

Daniel Abreu Marques: And France also has its own cybersecurity requirements for these systems. So sitting on top of the UN vehicle cybersecurity regulation and also the type approval requirements. So do you think this is that more general additional substance or a duplication?

Aryldo RUSSO: This is a kind of the should be complimentary one, but. When we have the vehicle that is already certified in UN 155, for example, it's vehicle centric certification. So we have to see if the other side is as well as good as the vehicle and so on, that there's different rules that they have to follow or they might follow different rules. And then we can check the cross acceptance of each one and something like that. So normally it has to be complimentary. One problem that we can see is that the homologation process is not quite transparent. At the end, what we receive is just a piece of paper saying, okay, the vehicle is homologated on this, this, and that. But what it's missing for us is a bit more of material that we can check what was verified and what are the constraints that has to be considered for the certification to be valid. So this is something that is missing in this case. Maybe we can, we might re-quest same things that will be already evaluated before, and then it will be a duplicated job.

Daniel Abreu Marques: And now we have talked a lot about proof. So maybe let's also talk about what actually counts as proof. So what is evidence in your world? Is it simulation runs, scenario catalogs, real world kilometers, field data from other deployments? So when you sit down to assess what convinces you and also what do teams bring you that turns out to be worth very little?

Aryldo RUSSO: Yeah, everything can be an evidence in such a way. But what is important is that we are looking for evidence that cover the safety requirements. So the idea is to have the full traceability. When you talk about the safety requirement, there is a kind of preliminary hazard analysis or risks that are performed. And in that case, there are several safety requirements that are listed that has to be proved true to cover the scenario, to cover the hazards and to avoid the hazards and so on and so on. And in that time, we can say, okay, some of them can be done by simulation. Some of them can be done by tests. Some of them are pretty much the development process that has to be put in place to guarantee a certain level of confidence and so on. And of course, some of them. we have to put the vehicle to run several thousand kilometers to be able to recover information. Pretty much when we talk about reliability and maintainability and stuff like that. So the idea is that a lot of different diversity information that comes together and that help us to build our conclusion on if the system is safe or not. It is as deep as the evaluator thinks that is necessary. So if someone just arrived with a piece of paper, we can say, okay, this is written that the system is safe. Can I see why it's safe or what evidence that prove that is safe? And then we are going to check about, not convinced yet. Can I see the data that prove that this comes? It always depends on the confidence that we have on the information that we receive. So pretty much a... It means if someone just arrives and say, okay, my vehicle run already 1 million kilometers per day, whatever, something like that, it's not sufficient to prove that the vehicle is safe or not. Because maybe it's running just in a straight line for forever.

Daniel Abreu Marques: Makes sense and you also mentioned traceability. So probably there is a hard one. So a safety case assumes this to trace behavior back to the requirements and modern driving systems are increasingly yeah getting the behavior more from the data. So this end to end approach. how do you independently assess something you cannot decompose that way and Also, is the French framework ready for that or is it still assuming a system you can really take apart?

Aryldo RUSSO: So from one point of view, the traceability can be done in a white box or in a black box. I what is important at the beginning is that, okay, we have a set of requirements and we achieve this requirement in a way that can be proven. So we have several different methodologies in the middle that we can put in place when the system is deterministic, when the system is not deterministic, like if we have a real time AI that is running. and changing the behavior of the system. We do have procedures and methods that we put in place to evaluate what is the result and how can we supposedly guarantee that it's not changing the full behavior that was already proven. So there is mechanism to evaluate that, it's true that's not the easiest way to do right now. But what I can say is that at the end, yes, the traceability is necessary and we have to find a way to prove from the A to B how we reach there and what are the evidences that we are not going to do anything different that was proven before.

Daniel Abreu Marques: Makes sense and the EU regulation approves the driving system and also the vehicle and the French framework as we covered authorizes the system and the service. So where exactly is the seam between them and also maybe what falls into the gap?

Aryldo RUSSO: Yeah, so as we discussed before, even for cybersecurity or in the general point of view, when we talk about the vehicle, we are talking about just the vehicle itself or the vehicle and its ADS. And this is one thing. France wants to approve what came on the top of that. In the Optimal world I can say is that it will not have any kind of overlap. We can just receive what we have from the EU approval, cross-accept and came with the next layer. The problem again is that we still do not know how it will come. It's just a piece of paper with a lot of information together that we can use to cross a set and go beyond. I think that this is the bottleneck for the moment. I don't think that there is a lot of overlap, but how we can not re-request something that has been already approved, this is the point that we need to... to move a little bit forward and see what will be the results of each from each side.

Daniel Abreu Marques: And the Commission has said it wants to reduce fragmentation and harmonize deployment conditions across the member states. So if Brussels writes an EU level operational framework, should then the French model be the template in your opinion?

Aryldo RUSSO: It's a tough one to answer because. I think that the French one is a good one at least to take as a model, maybe not the final one. There are some points in the French one that is very time consuming and maybe not really necessary for that. Like the cybersecurity guidelines are very hard to cope and also the... the root demonstration that because there is an specific document that has to be prepared, it's called DASP. And then this one is also very, very, very time consuming and maybe the result is not as strong as it should be. And so it could be a little bit lighter, but I think that the French model is something that could be used as a starting point to build something that will be. use it for everyone. We cannot forget that this is based on an railway transportation model and is pretty much used by everyone.

Daniel Abreu Marques: And France also extended this framework from passengers to automated freight at the end of twenty twenty four and the guides were adapted through twenty twenty five. So what changes when you take the driver out of a truck instead of a shuttle? So different hazards, different evidence, maybe also different assessors?

Aryldo RUSSO: Yeah, different assessors sometimes, yes, because the behavior of a truck is not the same behavior of a small vehicle. there is some technical points that change a lot, pretty much the dynamic behavior and so on. On the other hand, there is not a lot of things that was changed because... something related to the freight like the loading and unloading and stuff like that is not covered by the guidelines for the moment. So it's pretty much really the core operation that is covered. So the principle is pretty much the same. The basics are the same, only the experts that are going to be used in some of the domains that we mentioned before might be different. So pretty much for the domain number five.

Daniel Abreu Marques: And can you give our listeners an order of magnitude so for a team hearing about for the first time, so how long does a full safety demonstration take?

Aryldo RUSSO: Yeah, so if we talk about the certification process, it do not take more than the development one. So it comes along, so it will be like two, three weeks later that you receive the final report. On the other hand, if we talk really about the demonstration part inside of the development team, because of course there is an internal effort that has to be put in place. to build all the documentation that's necessary. like you said, like we said, the DCST, DPS, DS, all of this stuff is not prepared by the certification board, it's prepared by the developer and by an independent team. this should come along as well with the development process. So we have a V cycle that should be independent, but the time that it takes should not take more than the development process. And the cost for that, I think we can say that it's a kind of an expert guessing, but it's a kind of a 5 % of the project.

Daniel Abreu Marques: Interesting.

Aryldo RUSSO: these three I guess in number.

Daniel Abreu Marques: Yeah. It's a rough a rough estimate, but that but that that helps a lot. Thank you. So there's also another argument where I wanted to hear your opinion on. So everyone says okay, Waymo has driven hundreds of millions of driverless kilometers in the United States without equivalent system level dossiers or Chinese operators are scaling super fast and Europe has built in the assurance layer that probably nobody else thinks is necessary. Also do you think that is a safety asset or a competitive handicap

Aryldo RUSSO: I think that they are both in fact, it's a handicap for the moment because it's hard to convince that it's necessary based on the evidence that Waymo for example are presenting. On the other hand, what we are talking about is really a public transportation system that we should clearly define the responsibility in each stage. So I do think that the methodology that's put in place is something that is feasible and necessary. Maybe in the near future, a kind of lightweight procedure like that, but the way that is done to evaluate the full system and put responsibility in the correct boxes, I think that is still necessary.

Daniel Abreu Marques: one last question before we end. So if you could change one thing about the French framework with no political constraints, so what would it be

Aryldo RUSSO: Hard to say. What I think that would be interesting is to create a much stronger modular evidence that can be used in every place. Right now it's really, it is still in Rioneri what we are doing. So we have to redo several times the same thing. So at the moment that we can achieve something that is being already done, some part something is better than another part and try to just. cross-accept this, it will be much easier for everyone to accept the way that we are doing, the need to put some framework that cover all the system. For the moment, it's true that we are doing several times the same thing for the moment.

Daniel Abreu Marques: it was really interesting to hear more insights from the French framework. So thank you really much, Aryldo for sharing all those insights with us. It was really great to have you on the show today. And yeah, looking forward to the next time. Thank you.

Aryldo RUSSO: Thank you very much for the interview and we are open to discuss with anyone that thinks it's necessary or even to criticize what is done. It's even better.