speaker-0: past two years I've been working at Google. I'm leading the AI ⁓ activities in ⁓ Google Silicon Hardware Group and I guess I'll explain a bit about that ⁓ in the upcoming minutes.

speaker-1: You mentioned ⁓ I would call it a rule or a l a line of the tw ⁓ ninety percent accurate POC line. What did you see when when you you know you wrote about that? Yeah.

speaker-0: At that point in time I felt I'm surrounded by ⁓ demos ⁓ showing different solutions that are very specific, very, you know, ⁓ intent, right, and showing fantastic results, but on the same time scaling them to production is something extremely hard. It's really hard today to have to expose all the relevant data for AI to take this.

speaker-1: So now if an agent writes the tests and another agent runs them, what is the independent check?

speaker-0: Great question. ⁓ I'm thinking about it almost on a daily basis.

speaker-1: Welcome to Wearing Flop Flops. I'm Ronan, your host, and today we have a special guest from Google. A Googler. Google designed its own silicon and run its own flow and then has to live with whatever it decides to use. ⁓

speaker-0: Thank you. Happy happy to be here.

speaker-1: ⁓ tell us who you are and what do you actually do all day?

speaker-0: so my name is TBL, as you mentioned, and I've been working in the semiconductor industry for the past ⁓ I would guess something like sixteen years today. ⁓ I've been working at Marvel, at Intel, ⁓ and at Google ⁓ in the past two years. ⁓ the first part of my professional career was about being a post silicon engineer and working on firmware, working on embedded software, right? But somewhere around Two thousand eighteen, it felt to me that ⁓ my growing interest in AI or machine learning as it was called back then, technologies brings me to a point where this is where I want to shift my career at. So ⁓ at the end of twenty eighteen, I started working at a group inside Intel. It's called the AI Solutions Group. And this internal group is doing amazing things. building internal solutions based on AI for different aspects of Intel's work. That was what my main work ⁓ w while working at Intel. ⁓ and in my recent position there, I managed the entire ⁓ product group for press silicon verification, post silicon validation solutions. ⁓ it might sound like yet another internal solutions, but we are talking about massive ⁓ database solutions, right? ⁓ running on enormous scale on a daily basis. And I learned a lot from my period of time there. and also had a chance to work with some great people. Since then, in the past two years, I've been working at Google. I'm leading the ⁓ activities in ⁓ Google Silicon Hardware Group. And I guess I'll explain a little bit about that ⁓ in the upcoming minutes.

speaker-1: Amazing. So let's let's dive into, you know, looking into all all this ⁓ experience. So nine years at Intel doing ⁓ AI silicon validation and and now it ⁓ you know a company that designs its own chips and leads AI. Well what what's actually different between those two?

speaker-0: Right. So I I want to start by saying that both companies are a great way place to work, but my position in each of these company is fundamentally different, right? Because when I worked at Intel, ⁓ as a part of the AI solutions group, we were basically managing a portfolio of products that were aiming to assist pre Silicon verification and post silicon validation teams. They were there was mostly product related. We had our products and we wanted to distribute them and we wanted to provide significant values to our customers across the company. And we are talking massive scale here. But looking at my position at Google, I am a part of the silicone team here, right? At Google Cloud. That means I am a part of the team and my responsibilities are different. Let me kind of have a quick review review w ⁓ on what I'm I doing, right? So I have basically six vectors of interest. The first and the one that I find the most important is to encourage AI innovation across the company. That basically means that I need to open bottlenecks, that I need to find solutions. It means that I need to distribute ideas and solutions and algorithms and tools across the company. across the individual engineering to help them do their job better, faster, and so on. As a second vector, I have the capability to identify certain solutions or bottlenecks that cannot be driven by the individual user and take these moonshots and develop them as a standalone product that can be distributed centrally. So I am also working on ⁓ some solutions that are being built centrally and being distributed from there to the

speaker-1: When you're saying ⁓ moonshots, what do you mean?

speaker-0: It means that we are talking about solutions that give a nonlinear value on top of what current flows has. Usually these solutions require deep understanding and knowledge of AI capabilities. They require robust infrastructure, database, and flows that it's really hard for individual engineers to do as a part-time of his work. The third vector would be to work with ⁓ Google DeepMind. So as you know, right, Google DeepMind is probably one of the best AI ⁓ companies, AI groups in the world responsible s for some of the most ⁓ innovative solutions that ⁓ were released in the past few years. And we are working closely with them to develop breakground technology and ⁓ apply research on pr the type of problems that we encounter when we develop ⁓ our chip design products. I think the first vector would be to work with other companies, sorry, to work with other groups inside ⁓ Google. Google has other groups that are developing ⁓ silicon such as ⁓ the the the group that developed the pixel phones, Waymo and other groups as well. So it's important for us at Google to create a an at an atmosphere that ⁓ encourage the teams to collaborate and share information and share ideas ⁓ between these different teams. ⁓ I think fifth vector would be working with the big ⁓ EDA vendors on how we can apply AI solutions into the EDA tools, how can we collaborate better? How can we maybe reinvent some of the processes? I think we'll cover some of that later on. And last vector is working with the startups and and we have amazing startups working with Google, working in the industry, bringing bringing really innovative ideas into the table. And we at Google want to encourage that. We want to to be a part of it. We want to help these teams. So all in all, a very wide ⁓ area of work.

speaker-1: ⁓

speaker-0: Yeah, at least for me Yeah, it's the best work in the world, ⁓ working in the best company in the world and I'm saying that unbiased. Yeah.

speaker-1: To be. Unbiased. so ⁓ so many questions about those. But let's start with something that that you you wrote and and talked about now. So ⁓ you mentioned that there's ⁓ dual engine strategy. Every engineer ⁓ would develop agents of his own as a developer, and central teams with more knowledge and more resources would do what you call the moonshots, the big things that a ⁓ an individual can't do. So Eight months into this process after you wrote that, ⁓ which engine is producing more value?

speaker-0: Yeah, they both do, because they basically solve different problems. When I'm talking about agent engineer as an agent developer, what I'm talking about is basically AI democratizes software development, allowing the engineers to create softwares and create solutions that ⁓ until maybe a year, a year and a half ago, we needed central teams to ⁓ build, to maintain and ⁓ To construct, right? So the fact that today people can act as local agent developers give them an immediate localized velocity gain. They can create better task plans, they can write script that debug their failures, right? They can do so many things that fall under the category of 80 20 rule, right? Like that's a good enough solution. It's not productive, ⁓ it's not in the quality of a product. But it gives them a lot of value in their day-to-day execution. But the fact is that you can decentralize everything. So central team needs to focus on moonshots we've mentioned before. That gives a nonlinear breakthrough ⁓ at scale, right? They can be

speaker-1: Talking about flows and maintaining the the flow and and the engineers just using it and you know with their own agents or something autogonal or different?

speaker-0: I think I think both because first of all, investment in AI infrastructure, whether that's the way that you structure your databases or ⁓ allow agent communication or data sharing or drug solutions or whatever, right? That's something that comes from central teams and that's definitely there. What I was referring to is actually building products that gives a nonlinear value. So That's an orthogonal ⁓ solution, as you mentioned, because we anyway need to build these infrastructures. That's not related. ⁓ but here and there we identify solutions that may come from an individual users, or ⁓ for us as somebody who sees the big picture and and see all the data, and then we see okay, we can invest there in building a flow that is non-trivial, hard to maintain, and can be scaled. And produce a non-linear value to the entire company. I do want to say that the fact that we are encouraging this dual approach allows us to basically have thousands of people trying out new ideas every day. And that's like an MVP, right? And ⁓ min minimal viable product, right? People are trying all these things that when I worked at Intel, for example, I needed to interview ⁓ engineers all the time to try and understand what are the difficulties, how their day to day looks like, what do we need to improve for them. But here they're already doing that and we can identify easily which of their amazing ideas are worth being productized at scale.

speaker-1: Amazing. l let's let's have another angle of of the AI and and you know when there there's actually the the design intent, right? So it lives kind of in in the flow, but ⁓ who captures it? Is is it the tools or is it the engineers?

speaker-0: I think today most of the design attack is trapped somewhere in the engineer's head, right? It is a combination of chat threads, incomplete documents, hallway conversation, personal experience, right? So ⁓ it's really hard today to have a to expose all the relevant data for AI to take decisions. And I think we need to shift our minds.

speaker-1: Why why is it difficult?

speaker-0: Because ⁓ a lot of the information that people are using to implement their ⁓ areas of responsibility are not well captured in a way that AI can read and learn from. And the data sizes are probably enormous, right? So you need to build the right infrastructure to allow exposing it to the right tools. And all in all, this entire infrastructure. was built or developed because it serves humans well. Right? It works to serve the capabilities of humans, which are not exactly the same way that we want to develop our AI solutions. If we want to enable AI solutions, we kind of need to shift the mindset from implementation details to intent. So the goal is to have an intent-driven design force where architects and designers, right? there describe not the final output the way they decided but the intent behind of it as well. Right? And if we are allowing the AI to understand what was the original intent and how this intent needs to be checked, it allows it to have ⁓ the code generated and test itself in numerous ways. We'll discuss some of them later on. While in parallel shifts the engineering ⁓ I would say day job, right? From writing every single line of code to define the exact behavior, the exact verification, and the exact ways that us as engineers want to imp to prove that this intent was met, you know. And this is a mind shift that we have to go through in the upcoming years if we want to really utilize the capabilities that AI brings to the table.

speaker-1: You mentioned upcoming years. what do you see a year, two years, five years from now in in in that sense?

speaker-0: It's it's something that happens already, but as you well aware, chip design process is a very long one, right? So we are working on numerous products. Some of them started years ago, start some of them starting today. So of course this is something that keeps on evolving, right? And new projects look different than previous ones in the way that they are being structured. Because we want to see how we are utilizing AI better and improving our final ⁓ solutions in different ways.

speaker-1: ⁓ okay. ⁓ you mentioned ⁓ kinda I would call it ⁓ a rule or a l a line of the t tw ninety percent ⁓ accurate POC line. ⁓ what did you see when when you you know you wrote about that? What happened that made made you, you know, draw the line?

speaker-0: What's Yeah, at that point in time I felt I'm surrounded by ⁓ demos ⁓ showing different solutions that are very specific or or very you know ⁓ intent, right? And showing fantastic results, but on the same time scaling them to production is something extremely hard. So I th in that point in time, ⁓ I had a lot of sessions with external companies and internal ⁓ team members, right? Showing something that works in 85% of the time. And they used to say, Looks how good it ⁓ this is this is really impressive, right? It looks really good. We can easily scale it. But the the fact of the matter is that a 90% accurate Python script is a great productivity boost, but as a a fundamental EDA flow. Right, or something that we need to use in chip design. Eighty-nine percent sorry, ninety percent echo rate is basically a multi million dollar ⁓ retry and ⁓ loss of a lot of time and efforts, right? So I felt that the gap between what you are doing personally for your own usage or what some of the startup companies that I encountered are showing. And a robust production ready EDA solution is actually widening, not short not not not becoming something smaller.

speaker-1: I'd like to double click on that. ⁓ I guess expectation wise. I mean, ⁓ are we talking about, you know, ⁓ the simple things of just running the tools or about the holy grail of ⁓ designing and then ⁓ testing automatically and then running regressions and seeing the failures, debugging automatically, ⁓ checking in new code, rerunning it, and you know, no human in the loop is is that what you refer to or something else?

speaker-0: So the holy grail, I think we can touch about that more extensively later on. But I was referring to different solutions that were basically about building a layer of agents on top of existing LMs and demonstrated these agents can do X or can do Y. I'll give you an example, right? There are a lot of demonstrations available in YouTube and in other platforms. showing how AI can ⁓ design a risk f ⁓ processor. Okay. But you and I and I think our listeners know that there are so many differences between demonstrating that you could do something on a well written spec and on a code that was developed roughly ten, fifteen years ago and is a part of all the LLM learning knowledge. And taking that into a development environment where things are hectic and documentations are changed on a daily basis and you have a lot of limitations and you need to integrate your solution into existing ones or you need to make changes on existing you on on existing products, that is a fundamentally different experience. And I felt that this gap is being I won't say like overlooked. but it being underestimate in the difficulty ⁓ it presents for us as engineers to close. So that's why I wrote this original ⁓ text ⁓ a few a few months ago, right? And I wanted basically to drive a change where we are building a solu solutions centrally, but Solutions that also allow a very strong personalization layer that will allow the end user to adapt it quickly to their needs and also allow different teams to learn from ⁓ experience of other members in the team. But in fact, you know, that also presents new challenges of quality and maintenance. So, all in all, my ⁓ feedback there was that yes. We are seeing a significant increase in specific use cases, but it's not easily scalable and we need to work on it. That's what I try to say.

speaker-1: ⁓ okay, so you know, ⁓ kind of a relevant ⁓ follow up question is that you know Google has a a lot of partnerships with many companies. ⁓ you know. ⁓ when it comes to AI and and you know the things you just ⁓ mentioned, ⁓ what makes sense to, you know, do and keep in house and what makes sense to externalize to Third vendors, ⁓ third partner I beat, vendors.

speaker-0: yeah, I mean, we are of course very interesting in partnering with ⁓ vendors because basically we care about anyone who has technology that makes our chips better, right? but but we are looking for ⁓ true differentiation mostly. Right in some of the areas what we we need is something very specialized and is very massive at scale, right? That likely nobody externally to Google will handle for us, right? It's it's simply not large enough ⁓ for meeting our expectations. But if an external power

speaker-1: Here, why why are you saying that? I mean, you know, l looking at the big vendors and you know, there are a a lot of startups handling various problems and you know solutions. ⁓ but so far the at least the big vendors were able to provide anything that chip design or the semiconductor industry ⁓ needed. Is is this changing?

speaker-0: Well, when I said that what I had in mind are let's say things that relates to the model itself, right? If you want to have a a better Gemini model to meet your expectation as chip design, right, then you need to have the full stack available for you. And I think we're going to talk about that in details later on in the in the show. I'm not under ⁓ estimating the capabilities of the big vendors. I'm simply saying that there are certain use cases that I cannot expect external customer to do for me as an organization as Google, right? What I am looking for in partnership can we tell

speaker-1: about you know an example of where you have to own the the thing because other other you know external vendors cannot do it as good as internal things. I I guess you know it's a question that is related to every big company.

speaker-0: ⁓ I don't know about other companies in that sense, but I can give you an immediate answer, an immediate example looking on Gemini, right? So Gemini, similar to other LLM ⁓ solutions available in the market today, is trained on enormous amounts of data. But we all know that the amounts of let's say RTL code available for training are not even close to the ones that have to to the Python libraries, right? Or the software examples. Right. So if we ⁓ as Google, right, we want to train Gemini in a way that will be more aware of the RTL audiences and the way that you need to run a physical design flow and so on, we need to expose this information to ⁓ Gemini as a part of the training, right? And this is something that is our most secret code areas. There is no way we'll ever expose it externally to the company. So that can serve as an example to one of the things that we are doing internally and we don't expect anyone to do for us. Does that make sense?

speaker-1: ⁓ today it does. Yeah. Yeah. And ⁓ you know, did you want to say anything else about that?

speaker-0: Yeah, I I I just wanted to say that it doesn't mean that if an we we are not looking to work with external companies or external partners that holds deep domain knowledge or maybe a very advanced technology or some mature tools, right? We we we do want to work with them. We do wanna do things together. but what I'm trying to say is that we don't expect the industry today at least, right, to bring all of these solutions. So we are taking a much active a much more active part in that ⁓ sense of developing tools and developing ⁓ solutions for our internal use case.

speaker-1: Yeah, interesting. And is Google using LAN LM AI solutions for chip design?

speaker-0: Yeah, so I think that before I answer about that, I I just want to take a step back for a second and explain about what is LLM in in our perspective and what can be handled by it. So basically at its core, right, LLM is is statistical model, right? It predicts the next word and it is exceptionally well for text use cases ⁓ and to predict complex scenarios, right? But it struggles with structural or relational or contextual data like synthesis net, right? Or connectivity graph. It it doesn't mean that it can't work, but it does a series of steps in order to to compete with it and to to be able to handle with it. And it basically flattening the data and it loses a lot of the information that the original graphs holds, right? And it might create hallucinations, it might create mediocre results, and basically it what it definitely creates is a loss of the engineering trust, right? In contrast, ⁓ we have a lot of classical machine learning neural network solutions that are able to preserve graph relations and and other aspects of the chip design, right? So for example, let's look on reinforcement area. Just just for for you to be aware, right? Like re reinforcement learning, you you don't actually tell the model what to do. You just create a set of rules and you give prize, so to speak, right, to the model if he succeeds and you're you're tapping it saying you know you're you're wrong if in case it's it's it's doing the wrong thing. And and

speaker-1: Training ⁓ based on like images or dog images, cat images and stuff.

speaker-0: No, no, no, no. I'm no no no. I'm talking about reinforcement learning. It's it's it's a branch of of AI, right, that tries to ⁓ train the network in a similar way that you you are teaching humans, right? You're show you're providing some, let's say chess, right? You are providing the rules of the game and then let it play and learn by itself, right? And years ago, Google had a very famous ⁓ competition, right? ⁓ on on on Go, right? The famous Chinese ⁓ originated game. And of course the story is very familiar, right? Yeah Google presented Alpha Go that was able to to achieve ⁓ a victory on the best ⁓ Go player in the world and Alpha Go was was trained on all the existing games that were available but in parallel right Google presented a solution called AlphaZero where AlphaZero only knew what the rules of Go are And then he went and learned different mah ⁓ methodologies of playing by itself. And after we'd finished the training, right, it easily ⁓ won the Alpha Go engine. So allowing the AI to learn about the world by itself is actually something really useful. So somebody back then at Google, right, thought to itself, Well, you know, Go is kind of like doing place in route. in some sense, right? You need to place different ⁓ silicon gates, right, and to place them on on a given surface. And then Google presented the alpha chip model, right, which which was an an RL model that learned bytes off, right, through trial and errors and rewards and penalties, as I mentioned. And develop placement strategy that alchems some of the humans ⁓ engineers and some of the use cases. So this engine that is not an LLM engine, right, was able to conduct ⁓ some of the place and route ⁓ assignments very efficiently, right? And without using

speaker-1: About placing ⁓ route or only ⁓ placing the the big macros

speaker-0: It started with placing the big macros and then it it continues to other stages of of the of the process and and the original article was was published I think a few years ago by by Google DeepMind. So yeah, there are a lot of examples where you don't necessarily need to use LLM to do the work that you need, either because it might be more expensive or less efficient, or it's simply good enough, right? Sometimes Good enough is really, really fine and you don't need to go through LLM and the ⁓ challenges is present mostly around hallucinations and misunderstanding of the intent and so on. And it's our responsibilities as central teams to identify these use cases and help the teams to build the right solutions accordingly.

speaker-1: ⁓ I would look at that as moonshots, right? Is is that the right category?

speaker-0: not necessarily because some of them yes, some of them are definitely a moonshot, but some of them are actually when somebody comes to you with a problem and says, ⁓ I I have an LLM based solution that is capable of doing that, it consumes ⁓ one billion tokens every one minute, but it works pretty fine. But then you can say something like, Okay, but we can just run a linear regression here or logistic regression. Right, that have been around for dozens of years, right? And it will provide similar results and it will cost a fraction of the money and a fraction of the compute needed, right? So what is the better approach there? So not everything is moonshot. Sometimes it's it's quite the opposite. Sometimes it's just saying we already have solutions that have been working for so long, let's use them ⁓ instead of reinvent the wheel.

speaker-1: Okay, I I'd like to shift and talk about responsibility in this ⁓ AI area. So, you know, an an agent writes an I RTL on everything passes and then the se silicon gets back and ⁓ it failed. ⁓ actually recently ⁓ report said that only five percent of ⁓ the silicones come back first time working. So in the postmortum. What do you do? It's kind of a philosophical question. I mean, how do you track the problem? Who is responsible for what happened and you know, how to learn from it for the next revision?

speaker-0: Yeah, I mean i if we're s sticking to philosophical discussion, right? I I think that in the same way that you don't blame the compiler if your C code has segmentation fault, you don't blame your agent if your RTL fails. Because in my opinion, the responsibility of the of the product and the quality of it remains the human responsibility. So the way I think about it is that AI is a new technology. That provides amazing capabilities for humans. And engineers need to learn how to sign off code that they didn't necessarily write line by line. And they need to do that by creating a trustable, robust, automated frameworks, right? So the responsibility remains with humans because they need to define the intent and they need to define how this intent is going to be tested and probably use a lot of mathematically ⁓ proving ⁓ capabilities to make sure that this code is working correctly and so on. But it's their responsibilities and we are going to make a significant mistake if we think that the fact that somebody else writes the actual code or even iterates through the flow to some extent removes the responsibilities for us.

speaker-1: Would that change a year from now, two years from now, five years from now?

speaker-0: No. I mean, it's really hard to predict. I think that let's look two or three years ahead. I'm pretty confident that that will remain very similar. What will happen in ten years? Who knows, right? Who believe that three, four years ago we'll be where we are right now?

speaker-1: Yeah, ⁓ indeed. It it's it's difficult ⁓ to to think in, you know, the the the curve of the ⁓ development. What what would happen, you know, five, ten years from now it's it's ⁓

speaker-0: And I actually but Olen, I actually think that you're making a valid point here. We can look on the past and see, you know, things like the first time that EDA tools started to be a part of the silicon development, right? So I I wasn't aware around, yes, but I I do remember people ⁓ reading about people saying, How can we control our output if we are not the one placing the gates and understanding exactly where all the blocks are going to be placed. Or let's take verification for example. Thirty years ago you still would have run direct tests, right? But at some point in time you're saying this is there are too many options, I can't converge. Let's move into semi random ⁓ verification environments, right? How do you know as a verification engineer that you covered everything? You don't control it, right? So ⁓

speaker-1: You do have test plans, you do plan on what you want to see and ⁓ cover it, right? ⁓ what's if you know, toggle ⁓ lots of things to see that you're done.

speaker-0: Right, but let's take it from a a little bit of a different angle. Sometimes you when you get into post silicon you find a bug and then you reproduce it in press silicon and you found find out that it could be re reproducible. It could have been found originally if you would have written the test bench a bit differently, or your ⁓ randomization would look like ⁓ you know, ⁓ weighted a little difference, something like that, right? It's still your responsibility. You're not blaming the randomization function. That that's my point, right? So I assume that in the same way that we adjusted to these technologies and these ⁓ methodology changes, we do need to adjust ourselves to the new methodology that the AI area era brings with it.

speaker-1: I'd like to you know shift the question a bit to you know ⁓ close but different ⁓ place. So you wrote in the past that you demand ⁓ one hundred percent deterministic reliability ⁓ from short verification ⁓ tools. And now if an agent writes the tests and another agent runs them, what is the independent

speaker-0: check. Great question. ⁓ I'm thinking about it almost on a daily basis. I think I think this is a critical risk. ⁓ I think that if a system checks itself, we are prone for a double error for sure, right? And there are a few strategies that we can adopt in trying to defend ourselves from these type of failures, right? First we need to use ⁓ fundamentally different underlying models when we are writing verification and we are writing the code, or using fundamental different models to check those, right? Well, I'm saying something like let's use Gemini models to write the RTL code and a Gemma model to test the verification output, for example, right? it's it's something that we are thinking about. I also think that the independent check needs to come or at least to rely significantly on deterministic tools. So if we can integrate formal verification, static tests, automated data flow analysis, mathematically mathematical new ways to prove the code does what the spec says, we are significantly reducing the risks of missing something out. ⁓ I think it's probably still not a good enough answer because I don't have one. And we might need to invent new technologies or new flows. ⁓ just ⁓ out of the top of my head, right? Maybe we can ask AI to generate significantly more tests than we've done before, shifting a little bit from being randomized into more direct, but AI has no problem in creating one million different direct tests and evaluate your ⁓ output. Or maybe we should think about different ways to verify at least the code locally, right? In a way that we are increasing the likelihood that what we wrote is indeed correct. But again, if you have other ideas I would love to hear about them because I'm thinking about that all the time.

speaker-1: ⁓ sure, let's let's take it ⁓ offline. And actually I have so many more questions that I don't think that would fit into ⁓ you know one episode. ⁓ how about we stop here and we'll be happy to have you another time and talk about all the other questions that I ⁓ prepared for you. So thank you very much for joining. It was a pleasure.

speaker-0: Yeah, same here. Thank you so much.

speaker-1: And see you all in the next episode. Bye bye.

speaker-0: Bye bye.