John J Richards

US Patent:

20190205703, Jul 4, 2019

Filed:

Dec 28, 2017

Appl. No.:

15/857599

Inventors:

- Armonk NY, US
Austin Zachary Henley - Memphis TN, US
David John Piorkowski - White Plains NY, US
John T. Richards - Honeoye Falls NY, US

International Classification:

G06K 9/62
G06F 15/18
G06F 17/24
G06F 9/451
G06F 17/18

Abstract:

Performance of a computer implementing a machine learning system is improved by providing, via a graphical user interface, to an annotator, unlabeled corpus data to be labeled; obtaining, via the interface, labels for the unlabeled corpus data; and detecting, with a consistency calculation routine, concurrent with the labeling, internal inconsistency and/or external inconsistency in the labeling. Responsive to the detection, intervene in the labeling with a reactive intervention subsystem until the inconsistency is addressed. The labeling is completed subsequent to the intervention; the system is trained to provide a trained machine learning system, based on results of the completed labeling; and classification of new data is carried out with the trained system. Proactive intervention schemes are also provided.