Speaker: Akshay Badola
Organization :
Akshay Badola Ph.D.(Deregistered) from the School of Computer and information sciences, University of Hyderabad.
currently working as Senior Project Associate on IoE robotics project
Date: 18 July Tuesday 3 PM
Title:  Some important current trends in Deep Learning
Venue: New seminar room(EB208)



Research Talk: Some important current trends in Deep Learning by Akshay Badola, Ph.D.(Deregistered). Date: 18 July Tuesday 3 PM. Venue: New seminar Room (EB208).



Abstract:
This talk will examine some important current trends in the broad area known as "Deep Learning". 
We will discuss /Attention/, its first occurrence in modern NLP, and how it evolved into Multi-Head 
Attention of Transformers. We'll go through intuition, full mathematical notation, and the relation 
between the two along with PyTorch code for MHA. Then a brief exposition in issues surrounding 
attention-based models will be given. We'll then discuss Regularization in DNNs and trace some 
important developments since 2012 when these methods came into vogue till current trends. 
After that, we'll discuss some innovations in optimizing large Neural Networks.