CONTACT US

X-ray News

Fireside Chat: How AI is Changing the Way we Make Things

This week’s Fireside Chat with the Xperts explores How AI is Changing the Way we Make Things.  Creative Electron’s Jonathan Jimenez uses a modern SMT line as an example of  how integrating artificial intelligence across an entire factory can improve efficiency and quality.

As “J.J.” elucidates the ways in which AI can transform electronics manufacturing, it’s easy to imagine how it will translate to all manner of production, regardless of the product.  And don’t worry, the Terminator isn’t coming to get you, but the factory that makes him might be on the horizon.  Check out the full line-up of upcoming as well as archived Fireside Chats here.

Transcript:

Dr. Bill Cardoso:

Welcome to another Fireside Chat with the Xperts. Today, I’m really happy to have Jonathan Jimenez, or J.J. Like we call him, our VP of software development hosting the chat. And the presentation today is how AI is changing the way we make inspections. J.J. Has a unique perspective as a software engineer and with a lot of experience developing AI and machine learning engines for X-ray applications. So with that, take it away J.J.

Jonathan Jimenez:

Thanks, Bill, for the introduction. Like Bill said, today we’re going to be doing how AI is changing things. And for this presentation, because a lot of the people training and are going to be from SMT, we’re going to use SMT as an example, but the takeaways are much broader than that. And you can see how your particular industry would fit in. There’s our contact details on the screen there in case you guys need to get a hold of us. And at the end they’ll be available again.

Jonathan Jimenez:

So basically what this presentation is going to be is a broad overview of the process. So we’re not going to get down to the exact math, but we’re also not going to skip over some important details. So we’re going to talk about the actual process to develop a system. And again, this is going to be for an SMT. What is the big goal here? So the idea is you may have systems in your factories or in your process that give you data. And for a long time, that was what everybody wanted, they wanted data.

Jonathan Jimenez:

And what we’re seeing now is that what’s most important is being able to take that data, convert it into useful information, something that can be acted upon and take action on that. And so the idea is that the users will be able to basically elevate what they’re working on from lower level work, gathering data, or looking at data to using the information, to take action. And then a little bit later, we’re going to move on from there. Like I said, here, this is a beautiful X-ray image, you can learn a lot from them, but the idea is we don’t need to look at them anymore. We want to get past that.

Jonathan Jimenez:

This is what a typical SMT line looks like today. So you see a printer and then we have our solder paste inspection, our pick and place to go to reflow and then we go to inspection afterwards, whether that’d be optical X-ray or both. And so to understand how we’re going to gather the data and then formulate action from it, we have to understand what data is available on an SMT line. So the first machine that’s going to give you real data in a system like this is going to be the solder paste inspection. So that’s going to tell us things like excess solder, solder bridges, or insufficient solder. Again, this is data that you, as a operator or as a engineer, may already have from your machine. And the idea is that we’re going to take these items from each of these individual systems, and then we’re going to be able to put it together. So your SPI machine may know the extra solder, solder bridges and insufficient solder condition of the board.

Jonathan Jimenez:

And so the next system is the pick and place. The pick and place is going to introduce issues. So for example, just like the solder paste process introduced excess solder, we can have a misalignment issue with polarity or an issue with a missing component in the pick and place. Then we move on to the reflow oven. The reflow oven can introduce through misuse or issues with supply that can introduce voiding, bridging, solder balls, excess solder, insufficient solder.

Jonathan Jimenez:

To go over a couple of these issues for those of you that may be in a different industry and you kind of want to understand what we’re getting this data from, most of the issue that happen here with the exception of the ones that happened in the pick and place. So that includes the excess solder, solder bridges, insufficient solder, and from the reflow oven, the voiding bridging, solder ball problems, excess solder and insufficient solder again, all these issues are going to stem from issues with the SNB solder. So this is the paste that comes to the factory and it’s placed into the solder printer or the solder paste machine.

Jonathan Jimenez:

And all these issues are going to be either problems with the design of the board where the stencil for the board is incorrect or the solder itself is a problem or the process for example, in the reflow oven, it’s going through too fast or too slow. But most of these issues are dealing with the same part of the process right there. All dealing with the solder. The pick and place is a little bit different, but it can also affect those solder related problems.

Jonathan Jimenez:

And so then as we move on, we get to the inspection area, and this is where it’s important to gather the data. So from the inspection area, whether that be AOI or AXI or preferably both, you can get information and statistics and do statistical process control based on the results of the data you’re getting from the AOI and AXI. But these results are going to be down to the machine. So traditionally you have your AOI results separate from ASI results, separate from your solder paste inspection and separate from any information that might be available to, for example, the pick and place.

Jonathan Jimenez:

So the idea here is that we can take the measurement data and figure out not only is it acceptable at this moment, but what’s the trend. Is it getting better? Is it getting worse? And again, this is available from reasonably new AOI, AXI machines. If you’re doesn’t have it, that’s a problem, but this is what we have right now. And so the idea is, okay, well, how do we process this information at the moment?

Jonathan Jimenez:

So currently a line manager is going to start at the beginning of the line and do his rounds up and down, up and down on the idea that they’re going to be checking the information from each individual machine. And then they’re going to be analyzing that data and deciding, is everything okay? If there’s an issue, what could be causing it? If you have a very good line manager, then this probably works most of the time. This line manager needs to be incredibly experienced because they’re operating or they’re managing a large number of machines. They’re all doing their own thing. They’re all producing their own data. And so if you’re lucky, if you have a very experienced line manager, it’s working at the moment.

Jonathan Jimenez:

So the problem is that you do require a very experienced line manager, to be able to do something like this. You can have issues where the operator running an individual machine doesn’t catch the problem, doesn’t flag, the line manager, that’s a problem that’s going to happen through training or through when an operator’s doing a repetitive task, it’s easy to forget what they’re working on or let details slide. But even if the operator catches it, the next problem is, well, can the line manager take all these reports that are coming from the individual machine operators, put that together and understand what the problem is? And so again, you need a high level of experience for your line manager to be able to do that.

Jonathan Jimenez:

So what is the idea here? The idea here is to take the role of the line manager and help him, help him or her and get the information gathering, the data gathering, and the creation of information and automate that part. So that the line manager can take information and then act on it without having to manually go through and grab all that information himself, all of that data. So the way we do this, the purpose of this talk, the trends that we’re going to in the industry and other industries as well is the idea of data fusion.

Jonathan Jimenez:

So here we have kind of a system. So we have data entering a function, and the function is going to put together the data that’s coming in and then give a result. And so you have information here. The example is AOI, SPI and AXI. Other SMT lines are going to be more complicated than that other industries are going to have different sets of data, but usually it’ll be something similar where there’s some sort of optical inspection or x-ray medical will have this. You can fit the data that you’re putting on the left there to whatever your industry might be. But the general ideas are going to be the same here.

Jonathan Jimenez:

So you have the information coming in from the different systems. And then this is obviously a simplification, this middle step here, where we have these different values of statistics and information. This is actually already at an important step, which is, do you have the ability to represent that data in a way that you can operate it like a function? And so then can take the fusion system, which takes all this information, and then we can get a result from that. And from that resulting information, you can make decisions, take action.

Jonathan Jimenez:

So what lets us do this? So everybody’s heard about big data. And for years, there was big data before there were AI and AI is the problem. The solution to the problem that is big data. And big data is the solution to the problem, which is, what’s happening in our process? So you can’t have one without the other it doesn’t help if you have gigabytes and gigabytes of data, if you don’t know what to do with it. And so what’s allowed us to make systems that take this information and make produce information and make it take actions on it is a low cost GPU and advancements in machine learning. And so the idea is we can build a system has lots of compute power that can take in all this information that I showed you on the previous slide. And then it could pull meaning from it. So for example, understand how the data is connected from one machine to the other, how the data is connected from in the time series. So how it’s connected the data from an hour ago, it’s connecting to the data coming now, this data that’s coming out now.

Jonathan Jimenez:

And then the idea is we’re not just looking at one piece of data and then another, and then another separately we’re taking all the data in, and we’re making decisions based on all of it at the same time. So for example, if you have a solder paste inspection and it has an egregious failure, sure you fail it, you do that now. If it looks great, you pass it, you do that now. What you don’t do is, is a solder paste inspection results in a marginal results. And the AOI, that same area results in a marginal results and the AXI results in a marginal results, you are currently not, or at least most customers do not have a system whereby they can see, “Okay, there’s actually something wrong here. We’re not finding it. It’s not a usual failure that our machines can catch easily, but they’re detecting the side effects.” And so what we can do is take advantage of the fact that together that information is more valuable to figure out what is happening underneath? What is actually causing these differences in results?

Jonathan Jimenez:

And again, for a BGA, let’s talk about how this might work. So the AXI can automatically identify a bad BGA, for example, if it’s lifted egregiously. But if we take the AXI and the SPI and the AOI data together, we can get a much better understanding of what is happening to that BGA. So for example, from the AXI, we might see that there is a short and the SPI so that there was slightly too much solder and the AOI can see that it’s lifted. So now we know, “Oh there’s probably an area that has excess solder.”

Jonathan Jimenez:

Now that sounds like something we can do now, but the next big step is taking all that information for every component the entire time. And then over data from earlier in the day to later in the day, and putting that together to figure out exactly what the problem is going to be, or where is it heading? So you may have this BGA that failed, or you’re going to start seeing components nearby fail because these issues are going to start manifesting themselves more aggressively as this process gets worse and then it’s going to cause more errors. The idea here is to find them before they happen. So we take the extensive data set of all the issues. We have partnerships with SPI and AOI companies, and we can take care of the AXI information and data gathering and analysis. And so together we can understand where the trend is going and the data.

Jonathan Jimenez:

And so here’s an example. It kind of shows what conceptually is going on. This only shows three degrees of freedom, three, three dimensions. And so here we have boiled down the process of making it a PCB assembly to three things, solder paste, the actual solder paste quality, the printer and the oven. And obviously there’s a lot more to this, but this, boiling it down to three variables, makes it a lot harder to see. So you can see that for this one. Our system has determined that the solder paste is doing really well at this time. The printer has a few minor issue, but the oven is causing problems the oven is causing defects in the samples. And so this is what it might look like if we were only looking at three degrees.

Jonathan Jimenez:

If we move over, you can see what it might look like over… So we can see what this might look like when we add time. So instead of making it a spatial dimension, instead of adding arrow here, we’re splitting it up over three different times. So you can kind of see how it’s changing. So you see if you look closely, you’ll notice that the printer does not change at all. So the printer stayed in the same position. You don’t expect it to change too much, but we’re not having any more issues. So it wasn’t doing great in the morning, but it’s keeping steady.

Jonathan Jimenez:

The oven is the same. So it could have been a lot better. If this system we’re running like this it’s because the operators or the line manager decided that this was acceptable for the board that they were building. But you’ll notice the important part here that the solder paste is getting worse.

Jonathan Jimenez:

And so something is happening from 8:00 to 10:00. So it could be that they opened up a fresh bucket of solder in the morning, and it’s just drying out, it’s getting worse and worse throughout the day. It could be that their air conditioning broke and the humidity conditions aren’t being kept. So it can be a lot of different things. But the idea is that we figured out that the solder itself is a problem here. It’s not the oven, it’s not the printer. The solder is getting worse as time goes on, and we can detect that. We can let the line manager know, “Hey, something’s wrong. We’re not storing the solder correctly. It’s getting worse.”

Jonathan Jimenez:

All right. And then the, so we have this information, we have the system, we understand what we can do with it, but what is that going to look like for an actual process? What is that going to look like for taking this information and making something of it? So you can imagine that at first, we’re just going to have reports to the line operator. And this is something that is a lot closer to kind of like an assistance role. And so the idea, this is what I had spoken about at first, is we take the load of analyzing all the data off of the line manager and we allow the line manager to manage the line. We allow them to make decisions about what needs to happen based on the information that we’ve given them.

Jonathan Jimenez:

But what they don’t have to do, they don’t have to run from machine to machine to try and figure out what each of them is saying. At this point, they have this information, they know that the solder paste is drying out. They know that they now need to figure out, first of all grab another bucket of solder paste and switch it out. And now they can focus the next 20 minutes on figuring out, “Hey, what’s going on with our solder paste? Why is it drying out so quickly? Are we not putting the lid back on? What’s happening?

Jonathan Jimenez:

The next stage is going to be a self-healing line. And so the idea here is if we can detect that, for example, the reflow oven is going too slow. So spending too much time at each stage, then we can talk back to the reflow oven and say, “Hey, speed up a tiny bit. We’re having these issues.” And obviously this would all logged. A lot of times in the in between stage, we’ll learn to be able to recommend this and then have a line manager sign off on it. But then that way they can just make a decision. And then at some point we want to get to the point where this is all happening automatically. So we’re sending that information back and the machines are responding to it. And this doesn’t go one way, it can go both ways, from the right side of the, of the line, to the left side of the line, or from the left to the right.

Jonathan Jimenez:

So it could be that solder paste inspection says, “Oh, this this area here, it’s passing. It’s good. But actually, can you just make sure that you double check it?” Or, “AOI, can you make sure you double check that area? Just because there is a small deviation and we need to gather more data about whether or not this is going to be an issue.” And so you have the line talking back and forth, back and forth to the point where you can get it to minimize the error. And again, this goes back to that minimization problem, trying to take all these inputs and get them to zero, get all the issues and the different dimensions down to zero.

Jonathan Jimenez:

So there’s no reason why we have to stop at three and the systems that we have don’t all look at three. Here, you see, we added a fourth spatial dimension. So we had the oven, solder paste and the printer, but now we’re looking at the inventory. So we can do things like using an integrated parts counter system. You can go ahead and see, “Hey, is there something wrong with these parts that are coming from on the manufacturer? Are we seeing discrepancies when we X-ray those?” Or, if we have it in the inventory system, what solder paste is being used for each system?” And we can detect, “Hey, this solder paste from this manufacturer is consistently giving us worse results than these other ones.” All because we’ve been tracking it down to which materials were used during a process. Then we know what we want to buy, who we want to buy it from and how to over the course of weeks, days, months make the process better.

Jonathan Jimenez:

So again, right now we get data over the course of hours and decisions are made over the course of hours. With data fusion, we want to make decisions over the course of the line speed. And then the idea is we can also take those decisions, month long decisions about like switching suppliers and we can get information within days to say, “Yeah, this is a clear way to here.” So it’s all about taking the information and changing the scope.

Jonathan Jimenez:

And again, so what’s the point. One thing is to speed up. Another thing is to have a higher level understanding of our processes. So you see on the right here, this is assembly. And so in the past, we used a lot of assembly. If you use assembly nowadays and that means you’re doing some really low level stuff, most software engineers won’t be doing that. And so the idea is as software engineering has gone up and up and up in abstraction, we’ve gone past actually wiring things up to assembly up to C level languages. And we worked our way up to artificial intelligence, where we have a much broader handling of the issue where we give it the data, the expected results. And then we try and build a system that we don’t really care too much about exactly how it works, as long as it understands what’s going on, and it can give us results. Just like that, we want to move our Smart Factory up.

Jonathan Jimenez:

So we’ve gone from hand assembling everything to machines doing it. And as we work our way up, we want to have that just like we are automating the process of building the products we want to be able to automate the understanding and the data gathering from that whole process. And that’s how AI is going to help with that, that is what data fusion is going to be for the Smart Factory. So this is what kind of combine all that data and give us the gains that we got through automation in our understanding of the process. And, yeah, so that is a an overview of what a system looks like and what we’ve deployed. And I think right now would be a good time for questions if anybody has any.

Dr. Bill Cardoso:

yeah. I have a question here, JJ, from Scott. And he is asking about, sorry about that. So who’s asking about digital twins. So he’s asking about, how do you see the ability that ability of EMS managers to abstract manufacturing once this new techniques are implemented? So, digital twin is the ability to create a fully digital representation of the manufacturing line, the one that you showed earlier in the first slide. The ability to replicate that whole line as a digital entity. So you can literally copy and paste and have the same manufacturing line in Fremont, California, San Diego, or Hong Kong. So how do you see that enabling this abstraction of manufacturing going forward?

Jonathan Jimenez:

Yeah, that is right. I mean, you set that up beautifully. So the idea is that if we have a very sound understanding of the system, we know how it connects. We know which inputs cause which outputs. And we know the exact settings that everything has to have, and we have all the underlying information gathering built in, then that allows, for example, an operator to switch from one factory to another. So let’s say one factory has one set of machines, another one has a different set of machines. The machines do roughly the same thing, it might be from a different manufacturer. But if they are set up with the same data fusion system, then they’re going to have that same dashboard and they can get the same information, the same understanding of the process that they would have gotten in factory A at factory B.

Jonathan Jimenez:

And to them, they are just looking at the overall process, So they don’t have to go through the trouble of figuring out, “Where does the solder paste inspection machine that factory A has this piece of data at this location, but now I’m at factor B and I need to figure out, Oh, this is a different manufacturer. And I really just need to know this information Because I have to make a decision. But now I need to call tech support or somebody who might be at that factory permanently to get that information.” We don’t have to worry about that. At this point, at this level of abstraction, because we’re understanding the process at much higher level. The line manager, for example, is going to just be looking at the information coming out of the systems. It doesn’t matter to them, how that comes out of the system that’s what the data fusion is about. And so that does allow for portability both from the workers and then also getting that data and integrating it into the company.

Jonathan Jimenez:

So, There might be a company with 20 lines distributed across the US. And so the idea is if they’re on the same data fusion system, then they can have information from all of those locations available to management and the line managers, and they can switch it around. But the information is the same because it doesn’t matter what setup you have or what country you’re in. If your solder paste is too dry, then the information your solder paste is too dry, is portable.

Dr. Bill Cardoso:

Perfect. Matt has a question for you, J.J. He’s asking, “Once this initiatives are put in place and the strategies are executed, will it matter where factories are located?” In other words, will we have to pay the high rents in San Diego or Fremont, if you can plop the same factory in very low real estate locations, like no middle of Nevada or Montana?

Jonathan Jimenez:

Well, I mean, no. So you won’t need to worry about location as much as you do about the process. Because it’s really going to come down to the actual process. What this is going to allow is taking a known system, whatever location and whatever individual components you want, as long as you can the system and you know how to optimize it, you know how to control it and you know how to report it. It doesn’t matter who’s operating it and where it’s operated, one of the benefits.

Dr. Bill Cardoso:

Great. So that’s all the time we have for today. Thanks so much, J.J., For hosting this Fireside Chat with the Xperts. and pleased we have another very interesting chat next week with it’s going to be hosted by Griffin LaMosser about robotics and how we use it for automation of processes, specifically X-ray inspection. Thanks so much for watching until next time. Thank you.

Speaker 3:

Creative Electron.

Related Posts

Fireside Chat with the Xperts: Refurbished vs. Used X-ray Systems

When your budget demands a bit of belt tightening, new equipment may be out of reach.  And when researching the potential purchase of a preowned...

Read More
Fireside Chat with the Xperts: Refurbished vs. Used X-ray Systems

Partner Webinar: X-ray & Deep Learning-Based Inspection Solutions for Medical Devices

Our VP of Engineering, Carlos Valenzuela, will be joining Matt Remnek of Cognex in a conversation about X-ray & Deep Learning-Based Inspection Solutions for Medical...

Read More
Partner Webinar: X-ray & Deep Learning-Based Inspection Solutions for Medical Devices

Fireside Chat with the Xperts: Robots and Cobots – the Good, the Bad, and the Ugly

Robots and Cobots – the Good, the Bad, and the Ugly is the subject of this Fireside Chat with the Xperts.  It's no Spaghetti Western,...

Read More
Fireside Chat with the Xperts: Robots and Cobots – the Good, the Bad, and the Ugly