Computerized Grading
1 1

33 posts in this topic

Reading through the online copy of Jim Halperin's "How to Grade U.S. Coins" there's a chapter regarding computer-assisted grading.

"On May 16,1990 PCGS announced a major breakthrough in a computerized system that grades coins. The system, PCGS Expert, utilizes robotics, image enhancement, image processing and an online image database for its integrated computer system. The system will perform four primary functions:

1. automated computer grading of coins
2. computer aided grading
3. image archiving
4. digital fingerprinting

.....

On the positive side, computer grading systems can be highly consistent, often achieving rates as high as 90%. This is more accurate than any single human grader. By using digital fingerprinting, the service can keep records on all of the coins that they have graded. A "grading set" containing several hundred examples is far superior to that of one that contains one or two examples."

It would be curious if this was a source of some of the gradeflation that's seen when older slabbed coins are regraded.

Rather interesting since a few of us were talking of the merits of such a system recently.  Imaging technology has advanced greatly since 1990 but it's nice to know that there's a business requirements doc or plan out there already.  Somewhere.

.........

Interesting parallel, currently at work I'm dealing with the Arkansas Workers' Compensation Commission on a few things and noticed that they implemented EDI standards in 1995.  Which would've been cutting edge for the industry and made them an early adopter.  But then they just stopped and never did anything more with it while the rest of us have moved so far beyond what their capabilities are now.  It's unfortunate to see useful tech supported for initial development & implementation but left to wither & rot because nothing is dedicated to ongoing expansion of capabilities.

Anyway, if Mark Feld reads this he might want to ask Jim about updating the image links on his page.

Link to comment
Share on other sites

Computer grading would greatly reduce the profits and bottom line over time of any TPG, thus there is no incentive to adopt and use such tech.  Just one example where the hobby suffers to protect the profits.

Link to comment
Share on other sites

2 hours ago, Coinbuf said:

Computer grading would greatly reduce the profits and bottom line over time of any TPG, thus there is no incentive to adopt and use such tech.  Just one example where the hobby suffers to protect the profits.

Well, in a press release 10 years ago it was thought that a new laser imaging process would get us to that utopian.

"Imagine a world without coin doctoring. Imagine a world without 'gradeflation.' This is the right thing to do," said Hall."

https://www.pcgs.com/news/pcgs-announces-pcgs-secure-plustrade-service-for-increased-consumer-protection

......

My takeaway, PCGS coins prior to 2010 were not laser scanned so they're still ripe targets for gradeflation.  That's considerably after the OGH period ended in 1998.

Link to comment
Share on other sites

And yet notice that the tech Hall talked about is not in use, in fact we have moved away from grades that are biased on grading by standards (what little there ever were) into market grading.  And there is no room for machine grading under market grading, grading by a machine is all about technical standards.  However under the market grading process grading criteria such as strike and marks are given far less weight and instead gives much higher importance to eye appeal, a highly subjective component that cannot be done by a machine because its an emotional response.  I would love a return to a more technical grading with published and real standardized standards over the current "its got easter egg colors so bump it a grade" way things are done now, but I don't run any TPG companies so what I want or think is of very little concern.

Link to comment
Share on other sites

I hear you, Coinbuf.  I like attractive toning as much as the next guy.  But when I'm looking at coins for grading purposes I give more weight to strike and technical aspects.

 

Link to comment
Share on other sites

Computer assisted coin grading is highly applicable to the stuff put out by US and other mints as "commemoratives," bullion pieces and the like. It would also be extremely profitable by reducing human contact, time and error to a minimum. Those pallets of Silver Eagles and the like could be examined and slabbed in hours not weeks.

Such systems could also enforce standards for everything from proof-like to the mess of "uncirculated" pseudo-grades.

Link to comment
Share on other sites

This post probably cost me about 100 hours of fun :-)

I spent a couple weekends trying to do this following this paper: https://hal.archives-ouvertes.fr/hal-01795304/document --it's not all that difficult but the problem is that it requires a lot of high quality pictures per coin, per grade, per year, per quark (ie. 1909 VDOB) back and front--i estimate about 1M pix for US coin base all up.  Then it requires the user of the system to take the same high quality photos. There are big issue with normalizing glare of high end coins.  net is only a HA, or TPG has the base of photos.  It's pretty easy to take 1 photo and munge it up into many but the score goes down significantly.  Biggest issue I had is with identifying date--it's a much more difficult problem than identifying a license plate.

Note that PCGS has been doing ML "assisted" for a while https://www.pcgs.com/goldshield "With PCGS Gold Shield, each coin is imaged in high resolution, registered, and checked against PCGS’ vast proprietary imaging database. Our Gold Shield system benefits from artificial intelligence and machine learning, helping our graders quickly identify and remove counterfeits from the market."  Interesting enough they let their patent expire: https://patents.google.com/patent/US5224176

I'll spend a couple more weekends on it as it's an interesting rabbit hole for a geek that likes coins...

Link to comment
Share on other sites

On 6/15/2020 at 1:36 PM, Coinbuf said:

And yet notice that the tech Hall talked about is not in use, in fact we have moved away from grades that are biased on grading by standards (what little there ever were) into market grading.  And there is no room for machine grading under market grading, grading by a machine is all about technical standards.  However under the market grading process grading criteria such as strike and marks are given far less weight and instead gives much higher importance to eye appeal, a highly subjective component that cannot be done by a machine because its an emotional response.  I would love a return to a more technical grading with published and real standardized standards over the current "its got easter egg colors so bump it a grade" way things are done now, but I don't run any TPG companies so what I want or think is of very little concern.

Hall did say that the human eye is the best at grading coins at that roundtable interview with the four founders of PCGS. 

Link to comment
Share on other sites

4 minutes ago, Modwriter said:

Hall did say that the human eye is the best at grading coins at that roundtable interview with the four founders of PCGS. 

He did, but he also had a financial interest in PCGS and that doesn't make for the most objective standpoint.  From a company bottom line standpoint computerized grading would at some point in the future restrict profits as it would discourage crackouts and regrade opportunities.  The other issue is that grading has never been fully standardized, it has always been (and even more now) a wizard science that incorporates the least objective (and often the most weighted) criteria known as eye appeal.  Machines can learn but it would still be more predictable in this area than a human that is subject to many factors like; how does he feel that day, or how tired is he from being up all night with a crying baby.  All those human emotions and external factors do play a part into each grade, and because of that the opportunity for regrade and crackout submissions income that would not be a part of computer grading. 

Link to comment
Share on other sites

2 minutes ago, Coinbuf said:

He did, but he also had a financial interest in PCGS and that doesn't make for the most objective standpoint.  From a company bottom line standpoint computerized grading would at some point in the future restrict profits as it would discourage crackouts and regrade opportunities.  The other issue is that grading has never been fully standardized, it has always been (and even more now) a wizard science that incorporates the least objective (and often the most weighted) criteria known as eye appeal.  Machines can learn but it would still be more predictable in this area than a human that is subject to many factors like; how does he feel that day, or how tired is he from being up all night with a crying baby.  All those human emotions and external factors do play a part into each grade, and because of that the opportunity for regrade and crackout submissions income that would not be a part of computer grading. 

Well that's true for regrading and crackouts, but if the technology becomes as accurate as it can be, many people might trust the grading companies to finally be accurate. Copper coins would probably be last to be accurate, if ever at all.

Link to comment
Share on other sites

On 6/18/2020 at 2:19 PM, RWB said:

Computer assisted coin grading is highly applicable to the stuff put out by US and other mints as "commemoratives," bullion pieces and the like. It would also be extremely profitable by reducing human contact, time and error to a minimum. Those pallets of Silver Eagles and the like could be examined and slabbed in hours not weeks.

Such systems could also enforce standards for everything from proof-like to the mess of "uncirculated" pseudo-grades.

Fully agree that on stuff where eye appeal is not a factor that computer grading could work. However, I'm not sure if there would be any cost or time savings.

Computer Grading: The TPG pays someone to take the coins out of the tubes, load the coins into a tray, load the tray into the computer, waits for the computer to grade them, and removes the tray before it moves to the sealing room. I suspect that unlike the original computer that took 3 minutes to grade a coin, with modern processor speed and high speed cameras it could be done in a couple of seconds. 

Human Grader: Takes coins out of tubes and grades them in a couple of seconds and then puts them on trays based on grade. Off to the sealing room. 

The modern bulk graders aren't exactly paid a lot of money to begin with. I know one TPG hired a person who knew nothing about coins and trained them to grade ASE. Realistically there is only about 3 grades for them. 

Link to comment
Share on other sites

17 hours ago, bernard55 said:

This post probably cost me about 100 hours of fun :-)

I spent a couple weekends trying to do this following this paper: https://hal.archives-ouvertes.fr/hal-01795304/document 

That paper is interesting. I kind of chuckle that the authors have mathematical equations for grading, but I suppose they have to, for the analysis. I've mulled this over for years, and it seems like rather than pixel/image analysis, it could be something like LIDAR. Map the coin in 3D and compare to a "perfect" reference. I suppose that would be good for counting up dings and scratches, but have problems with weak strike versus wear, etc. ATS bought into some kind of optical focusing technique similar to this when attributing a pattern as "high relief" last year, and it didn't end well. There would be a lot of bumps in the road, and as gmarguli says, maybe not cost-effective.

Link to comment
Share on other sites

1 hour ago, gmarguli said:

However, I'm not sure if there would be any cost or time savings.

To parrot those at the beginning of the industrial revolution, a computer doesn't get sick.  It doesn't take vacation.  It can work continuously (save for scheduled update intervals).  It doesn't require salary nor benefits.

....

Glad to see that you've found joy in this @bernard55!  That patent they let expire is an interesting read.  I might be busy with that the rest of the afternoon.

Link to comment
Share on other sites

CRAWTOMATIC - their patent was written at a time when many of the current techniques were not known.  Things in tech have changed and made a lot of what's in there easier IF you have the coin in your possession (so you can control camera and number of images) and IF you build your trained model with enough high quality images per classification.  Anyone can hire the tech to build the model if you have the images.  It took me just a few hours to build the model for the identification on Tensorflow.  I am stuck on the date issue as OCR doesn't work well.  Even without building a trained model any 10th grader could build something that does an absolute diff on a threshold built from a bunch of high grade images (like in paper) as part of normalization prior to training.  I put my 10th grader hat on and did it in a few hours but then spent the rest of the weekend stuck on the date/OCR issues and the glare in the photo issue (both difficult to normalize for...)

image.thumb.png.068a9ab386a58c72d73e07200b389eea.png

Link to comment
Share on other sites

The issue with the date is interesting.  I feel like you could approach it from 2 angles, 1. the date is simply a design element (device) and should be graded as such, the machine doesn't need to "read" the date, or 2. the date determines the library of imagery to reference and grading parameters to work within, so the date needs to be read by the machine (or input by operator) to start the grading process.

If we approached the date as a design element then the grading across a whole series becomes standardized.  But I feel like that's not the case in practice, high 90s % feeling.  I know the Buffalo Nickel series best and the strike quality across years varies.  Some year & mintmark combinations, 1925-S for instance, anecdotally get better grade results while lacking in overall strike depth/crispness.  I believe I read a similar take on the Peace dollar series with the 1921 issue. 

Now, I get that in practice that means we're taking into account the quality of dies used for these YYYY/MM combos.  With a worn die the outcome is going to have a lower ceiling.  Or maybe it was a mechanical setting that year that lowered the pressure in the press. I get it.  But that certainly makes it harder to explain to a novice, or machine, why a MS-63 for one YYYY/MM does not share the same attributes as another YYYY/MM in the same series.

Or I could be wrong.  Not the first time.

Edited by CRAWTOMATIC
fixed a run-on sentence that was painful to read
Link to comment
Share on other sites

I think angle 2 would be the best. Either way machine learning is only as good as it's sources. If a human has to manually check everything after the computer, does the computer really need to be there. Maybe they should just stick to it being a "verifying" device used to double check actual graders rather than a single opinion, especially as a computer can only grade technically, and can't make a strong enough market grade.

Link to comment
Share on other sites

CRAWTOMATIC - yep, it has a lot to do with strategy and the goal of the system.  Is the goal for a third party (dealer) to use it, is it for a grading company to use it as the final word or is it to help the grading company in quality control.  if it's quality control, then overlaying a mask over it like the one attached below and grading only the anomalies in black can work and is pretty straight forward. I suspect that PCGS is already doing this given their early work on the subject.   There have been a few articles published about their work https://www.positronic.ai/blog/2018/5/10/pcgs-partners-with-positronic-to-develop-coin-grading-technology they even did a video on using it for counterfeit a couple years ago.  figuring out counterfeit if its a defect (misaligned or vampire bite marks) is really easy from photos.  what material its made of -- not so much :-)

 

image.png.801d6ba75095aac2a4abfda6b024da61.png

Link to comment
Share on other sites

but... if you have the right equipment (like an ortery.com camera -- pcgs owns a few -- see this great pix ) and you have the images (like NGC/PCGS/HA - categorized by grade) and you are in possession of the coin so you are in control of the camera then this is pretty straight forward and any dev that has some deep experience with tensorflow or pytorch could build it. 

Link to comment
Share on other sites

5 minutes ago, bernard55 said:

Is the goal for a third party (dealer) to use it

Forgive me, but I was only thinking of it's use from the perspective of a TPG to automate their grading process prior to encapsulation.  Or to provide that post-grading QA phase like Big Nub referred to.  For the record, I've always preferred two-pass blind verification systems in data capture processes - as if y'all cared.

But as an application outside of the TPG environment is tempting.  Like a tabletop box that could be used to grade a coin whether to sell it raw or determine likelihood of TPG results.  I use an HP G4050 scanner at home currently which is about 20x14x5.  The lighting is awful for coins but great for high quality photo & stamp renderings (4800x9600 dpi).  It's older tech and soooo slow on the highest setting.  If a coin grading box was created it'd have to be quicker and deeper to accommodate the multiple light sources - but not nearly as long or wide.  Then you'd have the created issue that while you could patent the technology you couldn't guarantee that these same end users decide to use it to slab coins themselves with the grades the coin box comes to.  Unless perhaps it's an enterprise application only and there's a user agreement where could they white label it.  "Craw's Grading MS-67, powered by Bernard's CoinBOX".  

I mean, that's a whole other thread I didn't even consider.  Dang.  Well, I'm off to open a shipment of coins I just got back today so I'll be pondering this.  Thanks, man!

Link to comment
Share on other sites

enjoy, if you want to continue to geek out--read this later https://www.frontiersin.org/articles/10.3389/fict.2017.00009/full   these databases are hard to find but there was really good work done on this subject by some clever people a few years ago. shows how hard it is to crack the code on the date issue but it has been done before.

Link to comment
Share on other sites

8 hours ago, bernard55 said:

how hard it is to crack the code on the date issue

As a layman this surprises me. It sounds like a database issue (I browsed thru your linked article). "Coinoscope" on my phone does a very good job of identifying the general series of coins. No problems picking out a Barber dime regardless of camera orientation, obverse or reverse. Once you've identified "Barber dime", you know exactly where the date is on an image, and there are 25 choices. If Facebook can pick out my face in a picture, it doesn't seem like a big deal to identify the date on a coin. Am I missing something?

Link to comment
Share on other sites

identifying a coin is pretty easy using something like tensorflow and just scraping the web for 5 to 10k images, categorizing them and then training on them.  like i said, i did that in a few hours (front of coin only) and have it up to about 80% accuracy if the photo uploaded is good.  going the next step (date/mint) is a bit more difficult but doable.  

Face detection then recognition is a well studied field.  here is some simple python that will explain the detection part using opencv https://www.geeksforgeeks.org/face-detection-using-python-and-opencv-with-webcam/  once you detect you can narrow against another training set. Using ML to do the same thing with coin dates is straight forward as well-- it just takes time to build the training set (cutting up hundreds or thousands of images, then categorizing them).   I just don't have the time or the patience to build that dataset.  That's why I was looking for one of the DB's outlined in the second article and they are not easy to come by for download.  Taking an easier road would be to do it with OCR and use Tesseract but I found that the font is a problem on most coins and it won't ID them correctly. You could train tesseract and I might try this in the future but I didn't go down that path. 

Link to comment
Share on other sites

You can get 100% accuracy with much less trouble with a 5-point matrix. Works for all coin types.  Use surface characterization and ultrasonic ranging to define deviation from nominal. (Ultra sound is more accurate at very close distances than light.)

Link to comment
Share on other sites

2 hours ago, RWB said:

You can get 100% accuracy with much less trouble with a 5-point matrix. Works for all coin types.  Use surface characterization and ultrasonic ranging to define deviation from nominal. (Ultra sound is more accurate at very close distances than light.)

I like this approach for a new perspective.  Instead of focusing on how to take a picture of a coin then program a machine to grade that picture we instead measure & map a coin and base the grade on that result.  There's a couple of methods available to measure reflectivity which could be used for PL, DPL designation I presume.  What about luster?  Is that a lower range of reflectivity?  Or a measure of diffused reflectivity?  

One gate could be needing a specimen coin for each series, year, mintmark on which subsequent coins are compared.  This may not always be possible with the early series.  As more "specimen" coins are mapped for a series you can identify standard deviations, revise the standard, and measure deviation from the standard.  Sub-grading:  strike, luster, a Prooflike scale, etc...

Edited by CRAWTOMATIC
Link to comment
Share on other sites

13 hours ago, CRAWTOMATIC said:

instead measure & map a coin and base the grade on that result.

And how would you compensate for differences in strike?  Otherwise a lightly worn well struck piece might grade higher than a MS  but more weakly struck coin.  Would you have to have different "maps" for each date and mint of a series?  For example you couldn't use the same reference "map" for a 21 and a 23 peace dollar. Or a 26 S and 38 D buffalo nickel.

Link to comment
Share on other sites

2 hours ago, Conder101 said:

And how would you compensate for differences in strike?  Otherwise a lightly worn well struck piece might grade higher than a MS  but more weakly struck coin.  Would you have to have different "maps" for each date and mint of a series?  For example you couldn't use the same reference "map" for a 21 and a 23 peace dollar. Or a 26 S and 38 D buffalo nickel.

That would be my presumption as well, that you'd need a specimen map for each date/mint.  That approach would apply if you're talking about the ultrasound mapping, photograding, or in hand grading, correct?

Link to comment
Share on other sites

DMPL, PL are very simple surface reflectivity measurements. Percent return/percent scatter = surface; bin range determines definition of "PL." "DMPL" is just a different range; likewise for proof surface (but not detail and other characteristics). "Luster" is surface scatter off linear irregularities caused by metal fatigue/stress. Can be defined the same way. Once a TPG establishes standards, nearly everything can be handled by the assessment system. The developer is not designing a system to be "trained" as one would an OCR - It is a compliance-with-definition system, and the defining coins can be as many or few as the TPG's developer wants to use.

The point of computer-assisted grade is not to grade everything - but only the great majority of coins with common characteristics. No individual date/mint mapping is needed - or even desirable. Put the expert people on what they do best - high value coins; and put the computer on what it does best - routine .

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
1 1