Christopher E Larson

US Patent:

20230108193, Apr 6, 2023

Filed:

Dec 12, 2022

Appl. No.:

18/079588

Inventors:

- McLean VA, US
Christopher Larson - Washington DC, US
Oluwatobi Olabiyi - Arlington VA, US

International Classification:

G06N 3/10
G06F 17/16
G06F 17/11
G06N 3/04

Abstract:

Aspects discussed herein may relate to methods and techniques for embedding constrained and unconstrained optimization programs as layers in a neural network architecture. Systems are provided that implement a method of solving a particular optimization problem by a neural network architecture. Prior systems required use of external software to pre-solve optimization programs so that previously determined parameters could be used as fixed input in the neural network architecture. Aspects described herein may transform the structure of common optimization problems/programs into forms suitable for use in a neural network. This transformation may be invertible, allowing the system to learn the solution to the optimization program using gradient descent techniques via backpropagation of errors through the neural network architecture. Thus these optimization layers may be solved via operation of the neural network itself.

US Patent:

20200265296, Aug 20, 2020

Filed:

Mar 5, 2019

Appl. No.:

16/293047

Inventors:

- McLean VA, US
Erik T. Mueller - Chevy Chase MD, US
Christopher Larson - Washington DC, US

International Classification:

G06N 3/04
G06N 3/08

Abstract:

Aspects described herein may allow for the application of stochastic gradient boosting techniques to the training of deep neural networks by disallowing gradient back propagation from examples that are correctly classified by the neural network model while still keeping correctly classified examples in the gradient averaging. Removing the gradient contribution from correctly classified examples may regularize the deep neural network and prevent the model from overfitting. Further aspects described herein may provide for scheduled boosting during the training of the deep neural network model conditioned on a mini-batch accuracy and/or a number of training iterations. The model training process may start un-boosted, using maximum likelihood objectives or another first loss function. Once a threshold mini-batch accuracy and/or number of iterations are reached, the model training process may begin using boosting by disallowing gradient back propagation from correctly classified examples while continue to average over all mini-batch examples.

US Patent:

20200265321, Aug 20, 2020

Filed:

Feb 14, 2020

Appl. No.:

16/791945

Inventors:

- McLean VA, US
Christopher Larson - Washington DC, US
Oluwatobi Olabiyi - Arlington VA, US

International Classification:

G06N 3/10
G06F 17/11
G06N 3/04
G06F 17/16

Abstract:

Aspects discussed herein may relate to methods and techniques for embedding constrained and unconstrained optimization programs as layers in a neural network architecture. Systems are provided that implement a method of solving a particular optimization problem by a neural network architecture. Prior systems required use of external software to pre-solve optimization programs so that previously determined parameters could be used as fixed input in the neural network architecture. Aspects described herein may transform the structure of common optimization problems/programs into forms suitable for use in a neural network. This transformation may be invertible, allowing the system to learn the solution to the optimization program using gradient descent techniques via backpropagation of errors through the neural network architecture. Thus these optimization layers may be solved via operation of the neural network itself.