speaker-0: Welcome to the second chapter of running flip-flops. I'm Ronan, your host, with me, Ron Elani, our co-host, and we have a special guest today.

speaker-2: Last time we talked about the verification wall, the huge chips and the long time it takes to simulate the chips and all those challenges. And today we have Claudio, our expert from Amazon. Claudio, can you introduce yourself in a few words?

speaker-1: Sure, first of all, thank you for inviting me and it's my pleasure to talk to you guys First of all, so I have been working in this industry in the chip industry for like 26 years since year 2000 and Been going through all the companies

speaker-0: Please 26 years old.

speaker-1: At least not far from there. And I started in 2000 as a verification engineer. I've been through several positions like technical leader and manager. And I've been in design and RTL implementation, post-silicon validation. but in general, I've been most of the time I've been doing press silicon verification, which is my main specialty and my main focus during my career in companies like Intel and Apple and Marvell Semiconductor and the startup in the meantime. And in the last 10 years almost I'm at Anapuna Labs, which is a part of Amazon and I'm enjoying very much doing what I'm doing. I think verification is a great area. to be specialized in, at least I think it's a combination between software and hardware and physical design if you want, we will talk about it. And basically you can see the full view of not only the chip design but also the system, which is something that is special to verification.

speaker-0: Well, amazing. ⁓ So, actually it blends with what we started talking about in the previous chapter, because looking at some public references, for example, the Graviton V. ⁓ So it's estimated to have about 132 billion transistors, and the next gen could be even greater, 180 billion transistors. And having said that, how much time tests are running? What do you do in order to test such a huge chip? So what methodology have you developed to co-op with such sizes?

speaker-1: Okay, so first of all, I think it's not one thing that we are doing in a certain way. It's several things that we are, several approaches that we are taking in order to deal with not only with the size, but also with the complexity and the timeline and also, you know, there are different kind of verification. If you look, you have the logical verification, which you want to be like correct, right? To want to check correctness, but it's also performance. When you come out with a server, must have, you must be, you know, become competitive in performance terms, not only in, you know, the cheap work. So it's good enough. It's not good enough. You have to, you have to cover more than just regular verification as people are, you know, using or ⁓ regular or maybe get used to do. But without you, even if I don't look at that, okay. And I look, okay, I say it's not only the logical. It's also the physical design, the design intent verification, which in most of the companies are, you know, less the focus of the verification. But in our case, we do want to get into every area. And we also look at the challenges, okay? We have architectural challenges, okay? You have several aspects of verification. have architectural, micro architectural, and physical design, which we call design intent verification, okay? So we start, Basically by setting a strategy. Okay The strategy it's per project it also can it's it's not but this is the big one Which is very high level right and you have per subsystem per unit or per block depends how we divide our verification because ⁓ You have to divide and conclude right? Otherwise if you do everything like Okay, I will do the unit level or I will finish the formal then I will start the unit level DV then I will Integrate it to the full chip and then I will go to emulation It will not take we have a cycle of two years from production to production This is what we have and in these two years we have to Open multiple threads. Okay that do have dependency on each one only on another but we want to minimize those dependencies. Okay, so we can talk about like reuse methodologies and stuff like that, okay, which cause dependencies. So we don't want, know, in my previous roles when I was in other companies, we have this reuse methodology, as you know, it was like a kind of belief, you know, it has to be reusable, otherwise, it's not good. So no, sometimes it's not good to have something reusable because you create dependency in both. layers okay in the unit level they have to be compliant to full chip and in full chip they have to integrate and we don't want for example integration of verification we want to be independent

speaker-2: That's a big trade off, a big dilemma in verification. The more usable you make the stuff, you're creating more dependencies. That's a good point. On the other hand, you don't want to throw away two years of work for the next project.

speaker-1: So we do have something that it's very, I think it's very specific, but it also creates dependency as everything that you wanna. So we use the, for example, the operational software that will run from block level, unit level to silicon. ⁓ Operational software, for example, let's say you have an interface. Let's say you have ⁓ a DRAM controller, right? And you have a driver. It will be used.

speaker-0: What do mean by professional?

speaker-1: exactly the same driver from the unit level to full chip, then to emulation and then to silicon.

speaker-2: So Claudio, just by the way, quick question. When you say the operational software, does that mean that you have the software like the assembly code or the C++ or whatever already on day one of the verification that you can load it into memory or run through a separate process with DPI or whatever? How does that work?

speaker-1: So I will not talk about how we do it technically, but I will do say, let's say it's a C code, okay? It's a C code, abstract C code, okay? For any driver or any embedded driver that we have, okay? Including the boot code. And we can run it in every environment. We have this, you know, mechanism that we can take the software and there are a few things that we are doing. So the one thing is we have the architectural modeling that... software guys can also develop the software in a very fast way because even if you take unit level or block level environment, it's still simulation and it's still much slower than architectural modeling. So they do use both things in parallel. They are not dependent. It means if the software guy didn't develop this on architectural modeling, it doesn't mean I will wait for him to finish there and then we'll start by unit. No, I will start anyway. Okay. And by the way, it's the same thing when we talk about unit level and then subsystem level and full chip. There is, don't, so we can work in parallel in full chip. We can in parallel bring up the emulation environment. At the same time, the unit levels will continue to work. We do communicate what it's working and what's not, what is not working at the detail level, but we do not stop anything. in parallel, everything must run in parallel. Otherwise, you know, we will have to wait, this will not work. Okay, we do concur and divide. But it's actually really, you know, really concur, right? It's not like, okay, we do it serial in a serial. ⁓

speaker-0: We asked our listeners for topics that they were interested in, that we talked about. And here's one question from Daniel Joffe. I hope I pronounce your name correctly, Daniel. ⁓ The question was, what's a good verification plan and how do you build

speaker-1: So, you know, there are many ways in doing the same thing and, you know, you can take every approach and can do it well. think the thing is not, ⁓ I will talk about what we are doing and then I will say what I think about anything else. Other companies or other people may have a different approach and it will work as well as this one works, right? So we have the approach that we, first of all, before we are doing any verification plan, we... do strategy plan. Strategy plan basically makes you think, how do I want to test the things? What are the environments? What are the platforms? What tools and what methods I'm going to use, I need to use, okay, in order to make this verification. This is not a test plan, okay? This do touches the main challenges of any subsystem unit or full chip, but it focuses on what exactly How do I divide the work? How do I ⁓ use different tools in different environment or in, know, how do I divide this ⁓ in different environments? How do they look? What VIPs do I use? What formal verification are going to be in this block or, you know, sometimes, for example, formal verification, not on a full block. They cover a part of the block, okay? And also in formal verification, you have to understand it's not always full proof. There are different approaches in formalization and you use them differently in different kinds of designs. don't use the

speaker-0: I from your point of view, so we're talking about verification, we're talking about formal, we're talking about prototyping, we're talking about emulation. Do you guys take everything into account in this strategy?

speaker-1: Exactly. That is the aim of the strategy the end of the strategy is to you know to make you think before you start to build the constraints and run simulation or build a formal it has to be You know connected it has to be synchronized. It has to be like multiple threads are running and you know, there is a the the thread that should get the scheduler that should Synchronize them. It's exactly the same

speaker-0: sake of understanding of the personas in the room. So for the sake of the discussion, in your current role, what is your responsibility and how do you make sure that what you plan and what the person next to you, if it's in relation, it's something else, how do you make sure that the strategy covers everything and there are no holes?

speaker-1: So when you do a strategy, it's not like Divi has strategy and Formal Verification has strategy and Architecture Modeling has strategy and Emulation has strategy. No, the strategy incorporates all this together. There is one owner, always one owner, okay, but he works with all the stakeholders and he's at the end of the day. For example, I'm in Graviton, the person that is responsible for the product, right? For the whole verification. It doesn't mean I need to, you know, I need to understand the full picture. I need to understand how, what platforms we use. Let's say about system level and, you know, subsystem. What subsystem are we going to, how are we going to split, how are going to split this chip in order to be able to run simulation effectively and not waiting for each simulation you can imagine. how long a full chip simulation can be in these huge chips. Without mentioning the numbers, you can imagine, right? So we have to.

speaker-0: about the strategy. Let's go one level down. What's a good plan then? You have the strategy in place. You know what everybody is expected to do. Now you own the verification, right? So what's a good verification plan from that point?

speaker-1: Okay, so there is something that you Through my career. I had a vision always, you know when I do verification I do not see only like the part of the verification that like we talked about at the beginning we have Architecture parts we have micro architecture parts and we have design intent parts Okay So the thing is that when you start to write you will test plan you have to ⁓ make sure right that you cover all these aspects Also the design implementation wherever you can okay. Sometimes you have IPS that you buy from a vendor You don't really have dive deep. Okay into the content and how they implemented but you do have to know Where what the different clock domains they have what recent domains they have you do have to know where the registers are located and you do have to not just Physically, I mean right you have to know where are having a where they have the memories and where they have the the five foes that you need to feel in order

speaker-0: that you're buying an IP from Acadian Synopsis, whoever, and then you go ahead and put it as part of the verification plan to actually test it.

speaker-1: Definitely definitely not always it depends. It's you know, we have to be very careful because not everything you can do that There are things that you cannot do that even if you want and it doesn't worth it, right? But in most of the cases where we can that's what we do because we are right accountable for the tape out I do not care. You know, I cannot blame anyone else if there is a bug and grab it on, has a bug, it's my bug. It doesn't matter that I can blame someone else. It will always be someone else that will blame, right? Because it's not me, definitely. But if I will look at the product or the project itself, it's something that in every place that we have the ability to do dive deep, you know, we have this principles in Amazon, think big on one hand, dive deep on the other hand, right? And then you have deliver results, and then you have insist on the highest startups. And this is something funny that, you know, we always run very fast during the verification. Then we get to, for example, freeze review or something like that. And we are become very, you know, we have this checklist that are very, very tough and very, you know, people are, you know, getting scared. But you said you want to you want to run fast. No, that doesn't mean that we don't want quality. So quality, right? It's not so you cannot say, OK, I will do whatever I can. and this is what I can. No, it's not enough. So, but let's go back to the verification plan because we talk about something else.

speaker-2: I actually going to ask, do you guys use any advanced tools for creating the plan, like AI, do you go like in the old days, I mean, you know, going paragraph by paragraph in the spec or sitting with the designer trying to understand the vulnerabilities and how does the operation of building the verification plan for such a huge chip work?

speaker-1: So if we talk about AI, for example, it's in the last two years, let's say. So we do have, we do use AI a ⁓ lot with documents. It's super helpful, but it does not solve you the problem of diving deep with a designer, understand the vulnerabilities, understand the architectural. And even if you take the block level, so I will talk about verification plan. are two things that you need to think when you do a verification plan. It's the what and the how, okay? It's okay. Yesterday I was in a verification review, okay? And there was something interesting because there were very features in the coherency and stuff like that, very complicated. And I said, you know, it's nice, but you know, another two weeks or a month when you will come to implement those, we need to understand how you do that because there is a very much importance. How do you check that? You say I check coherency, true sharing, false sharing, you name it, right? But how you do that? How do you know that you do the right thing, that you are really challenging the architecture, challenging the design? You have to be specific on how, not only the what.

speaker-2: So when you say how at that point, can you, for example, say, you know, this feature has to be tested in formal and not simulation.

speaker-1: It's not enough, right? It's still the what. It's the what and okay, this is strategy, right? In the strategy, you say, okay, this block or this feature will test in formal vacation, it's not the how. It's just the what and what tool we use for that. The how, it's what we are going to do practically, how the test would look like, how we are going to attack this problem in order to make sure that we do the right thing. And because if you do that and you come to a... a meeting and you explain the smart people that are sitting there what you're going to do then they can give you real feedback if you say okay i'm going to cover this item you know everybody's happy they will say good luck what a great test but it was but maybe it's not you know you are going to do something that is really not good enough it's never guaranteed you know how the in the verification i will never tell you that they ever finished the vacation in my life right it's not it's not something that can happen but i will do my best to know that you know to understand that i had at least tried

speaker-0: It's good advice. And Milani, do you have anything else you want to ask about that?

speaker-2: That's a good tip. I'm just curious in what format do you capture the verification plan? It's like good old Excel or do you have like something more fancy?

speaker-1: No, the most, the simpler one you can imagine. It's, we, you know, we tried not to focus on the things, you know, that on the, you know, there was one ⁓ guy that I interviewed once and he asked me, what Gantt tool you are using? And I was, what Gantt tool we are using? We don't. The what we are using it's not a gantt or you know, we have timeline of course and we have deadlines and you know, it's it goes without saying but the thing that I think that we are focusing on it's the mindset of the engineers and The mindset it's ownership, right? It's also another principle of Amazon. That's why I think I'm a poor very little a match Amazon culture you know all these principles that I I think that are really meaningful because If you think about ownership, right, if you interview someone and you will ask him what he would like to do, he will tell you, I would like to have ownership or something really important or a big scope, right? But not everybody understands what ownership means when it comes to practice, right? Because ownership, it's not easy, right? On one hand, you know, it's fun because you have a lot of things to think and to solve and to deal with, but on the other hand, it's a deal of, You like a ceo of your own task and you have to take it from the beginning to the end further I always give this example to my my guys I say, you know if your kid doesn't feel well and you go to a doctor And he still doesn't feel well You will go to another doctor, right? And you will find another doctor until you get the solution and you'll never stop to look for the doctor that will give you the solution

speaker-0: Basically and you

speaker-1: Exactly, you know, so that's what I mean. So the mindset, it's much more important than any tool that you will use, or any, you know, ⁓ gantt or format or whatever, it's important. I'm not saying you know, everything has a benefit, right? And you do automation, it benefits, right? Because you have like more time to do the real thing. So if you do, you know, the things that are like running regression and stuff like that, and so It really saves you time, but it's not solving your problem, right? It's giving you more like dealing with the real things.

speaker-0: Got it. So I think we talked a bit about ⁓ the verification plan, at least in high level and you know the expectations and how to do the strategy and how to really understand what's in there in order to.

speaker-1: So regarding the frequency, so it's the what and the how and another thing is how do you know that it is really done? ⁓ How you know that you know you did whatever you thought you did in the how right you did and you think that this how covers the what? But you need actually you know like a for

speaker-0: That's exactly what I wanted to ask you. When do you say I'm

speaker-1: Okay, so I will talk about that but I would like to add a point so each point that you have in the test plan you need to ⁓ to be To write also what is the criteria that say this is done? Is it coverage? Is it direct test and if it's direct test, how do you know that these direct tests, you know? Once what's written it can run and pass a regression It's still doing the same thing that you thought it is doing the so you have a lot of things you need to be sure that when you say something is done and covered, it's done and covered and you can prove it if you will go to a trial, right? And this is it. And the other thing now, how do you go to a table? So we have, first of all, we have a sign off, you have a long and the exhaustive checklist, right? But if you ask me, you know, you can go, I can go with you. Whatever check is you give me and I will still pass and everybody will be happy and nothing will be good, right? This is just to remind you this is just to remind you what are the points? You know that we learn from our lessons and then each project we do lesson learn right to understand what we need to learn for the next project and it's go a part of them go to the checklist not all of them

speaker-2: Do you have any objective measurement of when you're ready for a tape out like certain score for coverage or mutation analysis report or whatever?

speaker-1: So we have several, okay? We have first of all coverage. So we need to be at 98 % coverage, but it's not only the percent. I always tell the people, know, you can't...

speaker-2: I don't know that the 99 % is really 98

speaker-1: Right and also, you know a lot of people want no one wants to get and this Disappoint, you know in the in a review so you have to be we ask questions all the time in each sign off the sign off is not just you know, like a ceremony that gathers all the people and everybody's happy at the end ⁓ we did no no, no, it's it's a tough thing. It's a tough thing It should be tough and I will ask very annoying questions, you know And I remember when I used to do sign-offs, right? I was very angry that, know, why this guy is asking all these questions? Of course we did that. Why is no one thinking about that? But this is exactly how it should be. It should be annoying, it should be terrifying, and this is the only way it works. The other way that, you know, we are coming together and go over the check list, oh, everybody, everything is covered, we are all 98 % coverage, right, score. We have

speaker-0: Are you saying that the checklists are not enough in order to make a decision?

speaker-1: Checklists are enough if they're reflecting the reality Okay, but you have to make sure they are reflecting the reality and you have to ask the tough The only thing I I challenge I ask questions. It's not only me by the way. It's the culture, right? People are not going to these reviews just you know to To be to attend right they need to come and prepare we have cross check reviews before the sign-off that someone that is not related to this but has some awareness and know how to ask questions regarding the specific unit, full chip or whatever, it's not one sign-off that you do for the entire chip, it's a lot of sign-offs, right? And this is not one thing that the same checklist, yes, it's the same checklist, it's the same cross-check reviews, but for different units that you have, tens of them, tens of them.

speaker-0: the checklist is very long, but can you point, you know, a few key items must-haves that you guys are following?

speaker-1: Yes, so first of all, one of the things that we are doing that I think it's very unique, I don't know if, we do also software, software coverage as a part of our sign-off. So for example, if I will use a driver for the DRAM controller, okay, and let's say I used, and I run it as a part of the verification environment in all levels, but there is one function that, or one line that I didn't cover. It might be okay because you know this time is not used by the way a lot of we do it along with the software guys We are working on that together and it's safe for them a lot of code and do cleanups because we see things they have in in their code that is not used anymore and so it's really We do find things that we didn't activate a mode and you have it in the driver and say, okay Do we need this mode? Why didn't we have it in the coverage and you we ask ourself we do cross check

speaker-0: and how to do it. What else is you know? I'll do gate level SDF.

speaker-1: Okay, this is the other thing we let's talk about. We have three approaches here, right? We have the design, let's call it design or physical design aspects of verification, right? So we have also in RTL, we have this SDC simulations, we run SDC in RTL simulations, but we don't trust it completely. So we run gate level with SDF.

speaker-0: everyone to make sure that everyone understands. When you say SDC, ⁓ you mean that you're taking the scripts for the backend and the timing and taking them into the simulation and making sure that signals are getting disrupted based on what the actual hardware should be,

speaker-1: Exactly. Yes. For example, if there is a false path, right, it means that, you know, if the pulse fat or so output from some flop should never be used in any place anywhere, right, because it can be sampled with incorrect value or you never know. So this is something that is a seen easy example to explain to you.

speaker-0: any actual bugs that were captured with

speaker-1: Definitely, definitely. We always see. Because first of all, we start early. People are always asking me that are coming from outside the company and join us. They ask, you know, why do you need to run that? Because you have static tools that can check that, Check CDC and RDC and stuff like that. But we do it sometimes before the guys from the designer getting there, we already can find it in simulation. That's what we try to do, right?

speaker-2: the end of the day some of them are actually RTL bugs like if you didn't multi-cycle path you know it's an RTL bug it's not

speaker-1: Right? ⁓ It can be also an STC bug, by the way. If it's a false path that it was defined on something that is not a false path, you by mistake. Because someone put a star in the wrong place.

speaker-0: I think we reached the end of our chapter today. So I wanted to thank you very much, ⁓ Claudio, for being with us here. Let's continue that on our third chapter ⁓ together and we can follow up. You're on any last words for this chapter two?

speaker-2: ⁓ It's been a pleasure having Claudio here, we know a long way back and looking forward to the next chapter.

speaker-1: Thank you guys, it was really a pleasure for me to talk to you today and hope to see you next time.