Benjamin F Edwards

US Patent:

20200050945, Feb 13, 2020

Filed:

Aug 7, 2018

Appl. No.:

16/057706

Inventors:

- Armonk NY, US
Wilka Carvalho - Los Angeles CA, US
Heiko H. Ludwig - San Francisco CA, US
Ian Michael Molloy - Chappaqua NY, US
Taesung Lee - Ridgefield CT, US
Jialong Zhang - White Plains NY, US
Benjamin J. Edwards - Ossining NY, US

International Classification:

G06N 3/08
G06N 3/04

Abstract:

One embodiment provides a method comprising receiving a training set comprising a plurality of data points, where a neural network is trained as a classifier based on the training set. The method further comprises, for each data point of the training set, classifying the data point with one of a plurality of classification labels using the trained neural network, and recording neuronal activations of a portion of the trained neural network in response to the data point. The method further comprises, for each classification label that a portion of the training set has been classified with, clustering a portion of all recorded neuronal activations that are in response to the portion of the training set, and detecting one or more poisonous data points in the portion of the training set based on the clustering.

US Patent:

20190318099, Oct 17, 2019

Filed:

Apr 16, 2018

Appl. No.:

15/953956

Inventors:

- Armonk NY, US
Bryant Chen - San Jose CA, US
Benjamin J. Edwards - Ossining NY, US
Taesung Lee - Ridgefield CT, US
Ian M. Molloy - Chappaqua NY, US
Jialong Zhang - White Plains NY, US

International Classification:

G06F 21/57
G06N 3/08
G06N 99/00

Abstract:

Mechanisms are provided for evaluating a trained machine learning model to determine whether the machine learning model has a backdoor trigger. The mechanisms process a test dataset to generate output classifications for the test dataset, and generate, for the test dataset, gradient data indicating a degree of change of elements within the test dataset based on the output generated by processing the test dataset. The mechanisms analyze the gradient data to identify a pattern of elements within the test dataset indicative of a backdoor trigger. The mechanisms generate, in response to the analysis identifying the pattern of elements indicative of a backdoor trigger, an output indicating the existence of the backdoor trigger in the trained machine learning model.