CONTACT US

X-ray News

Fireside Chat with the Xperts: Surgical Tray Inspection

Automated Inspection Medical Surgical Kits

The Xperts are back at it, with their first fireside chat of 2021.  Who though surgical tray inspection could be so interesting, and the challenges so complex.   Host David Kruidhof, along with Dr. Bill Cardoso and Creative Electron’s Carlos Valenzuela discuss the challenges these surgical trays presented and the technologies deployed in solving them.

Because of their opaque packaging, X-ray inspection was natural fit, but seeing inside these sterile packages was just one part of the problem.  The inspection had to be thorough, fast, and highly reliable, and was made more challenging by the fact that the component parts within the trays were not always in the same location.  Their conversation elucidates the value of combining powerful technologies such and machine vision and artificial intelligence in successfully solving complex inspection challenges.

Enjoy the video, which includes some great image samples that help illustrate the problem and solution.  Reach out to us directly with any question and register for upcoming Fireside Chats with the Xperts and view our archives here

Transcript:

David Kruidhof: Welcome to another Fireside Chat with the Xperts here Creative Electron. Today I have with me Dr. Bill Cardoso and Carlos Valenzuela. So we’re going to be talking today about the automated inspection medical surgical kits, there’s a project we’ve been working on and have some exciting results to talk about, especially in light of recent events with a lot of more kits and things like that being used in this field. So welcome, Carlos. Welcome back, Bill. Good to see you guys.

Bill Cardoso:
Good to be here.

David Kruidhof:
So, Carlos, maybe you can talk a little bit about what these surgical kits are, why they’re important, why anyone would care about them?

Carlos Valenzuela:
Yeah. That’s a good start, right? So these can be as complex and as simple, and we’re not medical experts but we’ve seen some that are some sort of surgical kit for some sort of surgery, and there’s some simpler ones, maybe a COVID testing kit or a swab and little things here and there. But these are important because they do have key components inside of them, and most of the time they are not see-through. So they’re either in a bag or a pouch, or something like that. So that’s where we get involved, and then that kind of simplifies how we got into it and then why there was some interest in our technology for this industry.

Bill Cardoso:
Yeah, it’s good to know that a lot of time medical devices should be sterilized, right? So when the sterilization layer is for the most part opaque, so you can’t really see what’s inside and that technique that Carlos is going to describe it can be applied to any medical device or any sample, right? Where you need parts presence and placement to figure out where things are. Right, Carlos?

Carlos Valenzuela:
Yeah. And that’s one of the main kind of applications for machine vision is are all the components present? That is one of the… It’s almost machine vision 101. And in this case we’re looking at something a little more complex that might require different set of tools including your X-ray, because on some of our projects we combined optical with X-ray for other stuff, maybe barcode scanning or things like that, OCR, but looking the object is the main reason, looking inside this pouch that has been sterilized and not have to open it and make sure everything’s there, that’s kind of the main reason why we’re starting to deploy this sort of systems and these sort of projects.

David Kruidhof:
In this case manual inspection as Bill brought up is not an option, right? You have to open up these kits and make sure everything’s in there manually and then re-sterilize and reseal them. The cost is more or less not worth it, right?

Carlos Valenzuela:
I mean, your whole batch is a compromise.

Bill Cardoso:
Yeah.

David Kruidhof:
Yeah.

Bill Cardoso:
You have to make sure the person opening it and checking is not going to make a mistake, right?

David Kruidhof:
That’s true.

Carlos Valenzuela:
Mm-hmm (affirmative).

Bill Cardoso:
And get it sterilized again with missing something, right?

David Kruidhof:
Yeah.

Bill Cardoso:
Which-

Carlos Valenzuela:
And something kind of interesting to you here too is, you we’ve seen a lot of some recalls and some things like that happen. When you use our technology or any sort of technology similar to ours the idea is to protect yourself. So you’re taking a picture of a kid that’s been sterilized and you have the image with all the components inside, so if for some reason somebody says, “Oh, this didn’t have everything inside.” You have the picture with the serial number, the date, the time and all that. Who inspected it, when, so that is also a big part of this, us getting involved as the process starts or actually getting involved after something happens. So those are kind of the two important pieces, too.

David Kruidhof:
Yeah, with the surgical kit, for example, if they’re in the operating room the cost per minute, I don’t know what that would be, but it’s got to be really high, right? So if you open up the kit and you realize, hey, these doesn’t have enough swabs, right? Even just something relatively cheap in that component now someone’s got to go get that second one, five minutes. I don’t know, 10 minutes, right? If somebody is under anesthesia, there’s risk, there’s all sorts of big problems. These things really have to arrive in perfect condition and usable.

Carlos Valenzuela:
Yeah.

Bill Cardoso:
Yeah. The traceability point is also critical. Right, Carlos? So you can go back and check even at different points of your manufacturing or packaging process, where things are and potentially figure out what and where things went wrong, right?

David Kruidhof:
Yeah.

Bill Cardoso:
Yeah, and we like to get the call where people are setting up a process, right? And try to figure out where to use X-ray technology and this level of either machine vision or artificial intelligence to preempt these problems from happening, but we do have calls where people have a warehouse full of gadgets, devices, samples, right? And they know that a few of them have been compromised. Now you have to inspect everything 100, 200, 500,000 pieces of product to classify what’s good and what’s bad.

Carlos Valenzuela:
Right. Yeah. Yeah. That’s a good point. I think the inspection tools are becoming more important, not just X-ray, optical, any sort of inspection because it’s what closes the loop in your production line, right? If you’re missing a syringe on your kit, 3% of the time you have to go back to the syringe station and then see what’s going on. There is no other way to know it. There might be other processes, but creating this inspection tool that feeds back information into your production line that’s huge. So traceability and all that it’s becoming more important and actually becoming easier to do with our technology.

David Kruidhof:
Right. Yeah, not just easier but faster. Right. What kind of time does it take to do the analysis on something like this?

Carlos Valenzuela:
Just to put it in short terms, our machine is probably the fastest thing in the process. If you rely on a person to grab the bag from it, he’s going to be slower than our machine. Taking an image takes maybe half a second, processing it takes even less. So it’s all about material handling, it’s how fast can you put it inside the machine? How fast can you get it out? Some requires higher magnification, so you do have to stop for a brief second, take a picture, some more simpler stuff doesn’t even have to stop, more kind of what you see at the airport.

Carlos Valenzuela:
And we’ve done things a lot with some fancy stuff with conveyors, we’ve even gone into deploying robots and cobots outside the machine to grab that product. So yeah, I think we’ve kind of hit that plateau where everything’s so easy to deploy, robots, and I think people in this industry are kind of used to it. So I think our technology’s out there.

Bill Cardoso:
And that’s why when having the partnerships we have to be system integrators for a Fanuc and can Dorner and all those material handling and movement companies helps a lot so we can deliver a full solution not just a piece of it. We can really integrate a whole inspection cell instead of a piece of inspection process, right?

Carlos Valenzuela:
Yeah. And this industry is very well defined the automation that most of the companies we work with starting to have their own departments that are very familiar with our technology. Our robots, our conveyor, the communication protocols we use, it’s always becoming a very kind of seamless integration. Actually, machines used to be in this other room that just had a sign that says X-ray inspection and then you go in there and you X-ray now. Now it’s part of a process, we can be after maybe, a machine seals the bag or seals the box. We can be right after that and basically be seamless in your process.

David Kruidhof:
Yeah. And part of that too is explaining the safety aspects to a customer, right? Whereas you used to kind of hide it in a corner or put it in a separate room for safety, maybe it wasn’t necessary, but it was kind of this thought people would have, but the cabinets we make FDA has very clear guidelines about safety around it, right? Or we’re very comfortable with those handle that and it’s not a concern, so you can put it on the line with people all around it and they’re not going to be under any kind of safety concerns or unsafe condition, right?

Bill Cardoso:
Yeah, that’s where our expertise come using, right? To design a system that’s safe, right? With the locks and the proper amount of shielding so you can have it operating in any environment that’s needed. And those are the two questions we get often, right? It’s about safety of the equipment itself and if X-rays are going to damage your product. So safety, that’s our expertise. That’s what we do for a living, right? So we design machines and design systems to make sure that it’s fully safe and complies with all FDA and CE requirements. And then we have if X-rays damage the product. And yes, there’s no, right? I mean, I’ve been doing this for a long time and I haven’t found a case where our X-ray as Carlos said is very fast.

Bill Cardoso:
So the level of exposure is very small. And you got to keep in mind that the product you’re manufacturing, you’re packaging is going to be shipped somewhere, right? And other shipping companies have X-ray inspection systems that are much more powerful than the ones that we use for this level of inspection, right? I’m talking about the airports, shipping companies, border control, right? They all have very powerful X-ray machines to go through containers and bags, and pallets, right? So the level of exposure that we usually offer to the sample is very small and is fast, so it’s safe from both radiation exposure perspective and potential damage to the sample.

Carlos Valenzuela:
Yeah. And some of them also go through some sort of X-ray sterilization process.

Bill Cardoso:
Yeah.

Carlos Valenzuela:
That is a lot higher than what we do. So yeah, X-ray is not really going to damage product. In this case talking about kits, these are plastic pieces, they’re thin pieces, so they’re going to require the lowest power essentially that we can do. We do higher power for metals and castings, and things like that. But a medical kit is really going to require very low power that’s not kind of make a difference .

Bill Cardoso:
Yeah, and again from the people commenting on this video on YouTube I just want to make sure that we’re not saying that radiation doesn’t damage plastic and other materials, right? That’s not what we’re saying, but we do know that radiation is all about the time of exposure, the distance between the source and the sample, and the power of the X-ray source, right? So the three things combined give you the level of potential damage that radiation is going to have in the sample, and we do very low power in various tasks. So we do minimize the amount of radiation that the sample is exposed to.

David Kruidhof:
Right.

Bill Cardoso:
But keep an eye out for those commenters, man. You’re smart.

David Kruidhof:
Catch them early.

Carlos Valenzuela:
Yeah.

Bill Cardoso:
Perhaps.

David Kruidhof:
Yeah. So, Carlos, I think you brought some images of a kit we were working on. Correct, you can share those so we can take a look at them.

Bill Cardoso:
Cool. Sounds intelligent, nice.

David Kruidhof:
That’s right.

Carlos Valenzuela:
Can you guys see that image?

David Kruidhof:
Yep.

Bill Cardoso:
Wow, beautiful. Yeah.

Carlos Valenzuela:
Okay.

Bill Cardoso:
What are we looking at?

Carlos Valenzuela:
It’s a medical kit. Of course, we talked about it, it’s a project that we’ve been working on and it kind of has a combination of a lot of things that we’ve been discussing. It’s a combination of needles, tubing, there’s a scalpel there, just different components that they can move around the kit itself. I don’t have a picture of the package itself because the NDA is protecting this and all that, but basically, you can’t see through this object. It’s a white bag, even with the most powerful backlight, there’s no way that you can see it. With good results you might be able to see some stuff, but once you get on a different angle then everything gets really complicated.

Carlos Valenzuela:
I mean, you can see here how clear the image looks. You can see as components shift around it, you can know exactly where they’re at. And there’s two levels of what we do here or what we can do here, depending on what the client needs. Is location important? Do they have to be on the same location every single time? Is the doctor almost closing their eyes and just grabbing it or something like that, or do we just need a complete kit? Are they going to be open and they just put on a shelf or something like that. So this is where kind of our vision tools come in. Do we create a search region for the component? I can see here this one, the three vials in the middle are on top of each other, is that okay?

Carlos Valenzuela:
We can locate the three vials there and we can know their location, but does this merit a failed product? So it can be complex, I can see the device here, this one moved, even though it’s a complete kit, they’re kind of moving around. So that’s where we kind of work with the customer and what is their requirement? Of course, if this is normal throughout the process or through shipping or something, you don’t want to be failing a lot. So if this is normal in your process then we can accommodate for that.

David Kruidhof:
Right. You mentioned using machine vision as kind of inspection one-on-one to see if things are where they’re supposed to be. Is that the difference here with that versus using artificial intelligence, for example?

Carlos Valenzuela:
Yeah. Yeah. That’s a good question. I think, there’s the fine line of one to use the right tools. Now machine vision it’s pretty powerful with pattern recognition and location, and orientation, all that doesn’t matter, but there’s limits to it. There’s limits to what that can do. And then that’s when we look at artificial intelligence or deep learning just a system working through a database and some feedback. If you look on the right side there’s this tubing here, that is okay on many different locations and orientations, and that’s where maybe a deep learning would help for this type of inspection. If it’s flipped or if it’s coiled a little bit different, the system can start to learn what it’s going to look like.

Carlos Valenzuela:
And then sometimes X-rays can kind of vary in grayness a little bit, so the deep learning approach gives us that flexibility to find things even if they got darker without getting too technical into X-rays. There’s an effect called the parallax. So things might look different in different locations. So they might portray a different shadow, so using a deep learning approach where you’re feeding an information, the system is going to give us better results. So you’re telling the system, okay, this needle looks like this at this corner, it looks like this at this other corner and flipped. So it starts to understand what it looks like and what it’s looking for to give us better results. But to kind of summarize, our first approach is always try to do this with machine vision. I think it’s an easier path to the project, and if we hit a wall, then we start deploying our other more complicated tools.

Bill Cardoso:
Yeah. I think for this project specifically, the boundary between machine vision and AI is if you’re just looking to make sure that needle is in the right pocket, right? If it’s there or not, we can set up a region of interest and with machine vision we can figure out if the needle is there, right?

Carlos Valenzuela:
Yeah.

Bill Cardoso:
Now, if that’s the question it’s a machine vision path and pretty straight forward, we’re pretty much done. Now, if the question is, is that needle anywhere in the package, right? That became a much more complicated. I mean, exponentially more complicated problem that machine vision can’t answer. Especially if the needle is going to overlap with the tubing or is going to be on top of the syringe, it becomes a very complicated problem that again, machine vision can’t answer. And that’s where we deploy artificial intelligence, right? Because this intelligence we can train what a needle looks like, right?

Bill Cardoso:
And we can tell it… We can train AI to find a needle at different positions of the tray and over time, and with more samples as we look at more images, AI does a pretty good job of finding the needle in the haystack if you will. So it’s difference between the well-conceived problem that machine vision can answer in the ill-conceived problem which is what AI looks for, right? AI is ill-conceived, you can’t really describe in a set of rules exactly what is the problem we should try to solve. That’s where the boundary is, and this project is very clear. Depending on what the client’s looking for, we can deploy either machine vision or artificial intelligence. But the cool thing is that we can do both, right? So we can solve the problem no matter what the extent of detail that the client is looking for.

Carlos Valenzuela:
Yeah, and even combine both technologies in one same solution.

Bill Cardoso:
Yeah.

Carlos Valenzuela:
Because this is kind of the beauty and the problem… Everyone want to call it problem, but what X-ray gives us. If you look at the current image, you have a needle and vials on top of each other, and in an optical world, the product that’s on the top that’s the one that you’re going to be seeing. But in X-rays you’re seeing all of them. So a pattern match pattern algorithm wouldn’t work, because the object is actually now a new object, because you have two on top of each other.

Bill Cardoso:
Overlapping them.

Carlos Valenzuela:
Yeah. So this is kind of where that line that Bill was describing is giving the system the flexibility to be able to be matched with different configurations or with different type of views. You have one on top of each other, you’re still going to find it. So that kind of describes, but in this application X-ray becomes a very powerful tool.

Bill Cardoso:
Yeah. And it goes to the process we use right to solve those problems. We don’t come to the problem with a solution and we try to fit solution to the problem, right? We first start with a proper description of what the project is and what are the boundary constraints that we’re trying to achieve, and then we develop a solution to meet those constraints. Right.

Carlos Valenzuela:
Yeah. Yeah. And we do what we call a feasibility study. It’s customers reach out to us for a solution and sometimes we don’t know if we have it, right? We’ve have experience and sometimes we know right away which is the good match or not, but doing a study like this is going to give the customer or who’s ever reaching out to us the confidence ourselves too the confidence to move forward with some solution. So yeah, I think that’s a good point.

David Kruidhof:
Yeah. Those feasibility studies can be really critical. Carlos, we were working on a project, they were looking for 27 different plastic parts in their widgets that they made, they wanted to verify that all 27 of them are in there, but they’re all the same kind of plastic, right? So we did quite a bit of work with them, different angles of inspection, and… Hey, what can we see? What can’t we see? What should you just use optical inspection for, right? X-ray is not always the answer, and we don’t want to be pushing someone into that box. And Bill said we’re not pushing people into a solution that we have just because we have, let’s find the best one for the customer. So that was a really interesting project from the feasibility study stage of finding out yeah, you can see this. Oh, and that’s not there, then, maybe you can’t see that one particular part but you can tell that all this other stuff was changed because the thing wasn’t in there. So-

Carlos Valenzuela:
Yeah. Addition by subtraction or something, right?

David Kruidhof:
Yeah.

Carlos Valenzuela:
If this is not there, then we don’t even have to look for other things. That was a good project to kind of start deploying, also with our experience with machine vision and AI start also looking into… Even though X-ray is our expertise and that’s what we do 100% of the time start to involve other technologies, optical inspection and just kind of blend them into the same solution. Yeah. Maybe we could see 15 different plastics and components on one device and we couldn’t see the next one, but just putting a camera on the top was able to give us that view. So those are the things that we blend together to give a better solution to the customer.

Bill Cardoso:
Yeah, which is by the way nice set up, Carlos. So that’s the topic of our next Fireside Chat, which is data fusion. So how you can combine different imaging modalities, right? To come up with a full solution that’s actually better than the single modalities and all.

David Kruidhof:
Cool. And that wasn’t on purpose. That’s good. Give them a taste for next time. So yeah, we can wrap it up here. As Bill mentioned, we’ll be meeting again in two weeks to discuss data fusion. So look forward to report to that, and if you have any questions for us as always, please leave a comment on our YouTube channel and we’ll get back to you as soon as we can. Thanks, Bill. Really appreciate your time, and thank you, Carlos. Always good chatting with you.

Related Posts

The Semiconductor Shortage Hit the Fan! Now What?

When things get weird, we often look to experts for solutions.  The COVID-19 induced semiconductor shortage is no exception.  In this week's Fireside Chat with...

Read More
The Semiconductor Shortage Hit the Fan! Now What?

Fireside Chat with the Xperts: Semiconductor Shortage, Fraud, and Counterfeits

Host David Kruidhof, Dr. Bill Cardoso, and guest Richard Smith of ERAI join forces to produce "one of the top Fireside Chats we have ever...

Read More
Fireside Chat with the Xperts: Semiconductor Shortage, Fraud, and Counterfeits

CR and DR with IR (Supplies and Services)

Putting the "X" in Xperts this week, Pieter Medema of IR Supplies and Services joins Dr. Bill Cardoso and host David Kruidhof, to share his...

Read More
CR and DR with IR (Supplies and Services)
Skip to content