Machine Learning - Course Handout

BIRLA INSTITUTE OF TECHNOLOGY & SCIENCE, PILANI

WORK INTEGRATED LEARNING PROGRAMMES

Digital

Part A: Content Design

Course Title	Machine Learning
Course No(s)	IS ZC464
Credit Units	3
Credit Model
Content Authors	Vandana Agarwal.

Course Objectives

No
CO1	Machine Learning is an exciting sub-area of Artificial Intelligence which deals with designing machine which can learn and improve their performance from examples/experience. This course introduces the student to the key algorithms and theory that forms the core of machine learning.
CO2	The course will cover various machine learning approaches.
CO3	The course emphasizes various techniques, which have become feasible with increased computational power. The topics covered in the course include Regression, Decision Trees, Support Vector Machines, Artificial Neural Networks, Bayesian Learning, Genetic Algorithms etc.

Text Book(s)

T1	Tom M. Mitchell, Machine Learning, The McGraw-Hill Companies, Inc. International Edition 1997 (http://personal.disco.unimib.it/Vanneschi/McGrawHill_-_Machine_Learning_-Tom_Mitchell.pdf)
T2	Christopher M. Bhisop, Pattern Recognition & Machine Learning, Springer, 2006 (http://www.rmki.kfki.hu/~banmi/elte/Bishop%20-%20Pattern%20Recognition%20and%20Machine%20Learning.pdf)

Reference Book(s) & other resources

R1	CHRISTOPHER J.C. BURGES: A Tutorial on Support Vector Machines for Pattern Recognition, Kluwer Academic Publishers, Boston, pp. 1–43.
R2	K. Sastry, D. Goldberg, G. Kendall: Genetic Algorithms.
R3	http://elakiyaelamaran.weebly.com/uploads/2/1/9/2/21924368/genetic_algorithms_in_search_optimization_and_machine_learning.pdf
R4	Julie Main, Tharam Dillon and Simon Shiu: A Tutorial on Case-Based Reasoning

Content Structure

      1.            Introduction

                        1.1.            Objective of the course

                        1.2.            Design a Learning System

                        1.3.            Issues in Machine Learning

      2.            Mathematical Preliminaries

                        2.1.            Probability theory

                        2.2.            Decision Theory

                        2.3.            Information Theory

      3.            Bayesian Learning

                        3.1.            MAP Hypothesis

                        3.2.            Minimum Description Length (MDL) principle

                        3.3.            Expectation Maximization (EM) Algorithm

                        3.4.            Bias-variance decomposition

      4.            Bayesian Learning Techniques

                        4.1.            Bayes optimal classifier

                        4.2.            Gibbs Algorithm

                        4.3.            Naïve Bayes Classifier

      5.            Linear models for Regression

                        5.1.            Linear basis function models

      6.            Linear models for classification

                        6.1.            Discriminant Functions

      7.            Non-linear Models & Model Selection -I

                        7.1.            Decision Trees

      8.            Review Session - I

      9.            Non-linear Models & Model Selection -II

                        9.1.            Neural Networks

  10.            Instance-based Learning - I

                    10.1.            k-Nearest Neighbor Learning

                    10.2.            Distance-Weighted kNN Learning

  11.            Instance-based Learning - II

                    11.1.            Locally Weighted Regression (LWR) Learning

                    11.2.            Case-based Reasoning (CBR) Learning

  12.            Support Vector Machine - I

                    12.1.            Theory of SVM

                    12.2.            VC dimension

                    12.3.            Linearly separable data

  13.            Support Vector Machine - II

                    13.1.            Non-linearly separable data

  14.            Genetic Algorithms - I

                    14.1.            Properties

                    14.2.            Solving a problem

                    14.3.            Operator Selection Methods

                    14.4.            Basic Genetic Algorithm Operators

  15.            Genetic Algorithms - II

                    15.1.            Representing Hypotheses

                    15.2.            GABIL

                    15.3.            Hypothesis Search Space

                    15.4.            Population Evolution

                    15.5.            Schema theorem

  16.            Review Session - II

Learning Outcomes:

No	Learning Outcomes
LO1	Study and analysis of Machine Learning algorithms
LO2	Study of theory of mathematics usable in Machine Learning
LO3	Study and analysis of Supervised  learning techniques
LO4	Study and analysis of  Unsupervised learning techniques
LO5	Study and analysis of some applications of Machine Learning

Part B: Learning Plan

Academic Term	Second Semester 2017-2018
Course Title	Machine Learning
Course No	IS ZC464
Lead Instructor	Vandana Agarwal.

Session 1

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	1	Introduction Objective, What is Machine Learning? Application areas of Machine Learning, Why Machine Learning is important? Design a Learning System, Issues in Machine Learning	T1 – Ch1
During CH
Post CH

Session 2

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	2	Mathematical Preliminaries Probability theory, Bay’s Theory, Probability Densities, Gaussian Distribution, Decision Theory, Minimum Misclassification Rate, Information Theory, Measure of Information, Entropy	T2 – Ch2/other online references
During CH
Post CH

Session 3

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	3	Bayesian Learning MAP Hypothesis, Minimum Description Length (MDL) principle, Expectation Maximization (EM) Algorithm, Bias-variance decomposition	T1 - Ch. 6
During CH
Post CH

Session 4

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	4	Bayesian Learning Techniques Bayes optimal classifier, Gibbs Algorithm, Naïve Bayes Classifier	T1 - Ch. 6
During CH
Post CH

Session 5

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	5	Linear models for Regression Linear basis function models, Bayesian linear regression	T2 - Ch. 3
During CH
Post CH

Session 6

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	6	Linear models for classification Discriminant Functions, Probabilistic Generative Classifiers, Probabilistic Discriminative Classifiers	T2 - Ch. 4
During CH
Post CH

Session 7

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	7	Non-linear Models & Model Selection - I Decision Trees	T1 - Ch. 3
During CH
Post CH

Session 8

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	1-7	Review of Session 1 to 7	Books, Web references and Slides (L1-L7)
During CH
Post CH

Session 9

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	9	Non-linear Models & Model Selection - II Neural Networks	T1 - Ch. 4
During CH
Post CH

Session 10

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	10	Instance-based Learning - I k-Nearest Neighbor Learning, Distance-Weighted kNN Learning	T1 - Ch. 8 R4
During CH
Post CH

Session 11

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	11	Instance-based Learning - II Locally Weighted Regression (LWR) Learning, Case-based Reasoning (CBR) Learning	T1 - Ch. 8 R4
During CH
Post CH

Session 12

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	12	Support Vector Machine -I Linearly separable data	R1
During CH
Post CH

Session 13

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	13	Support Vector Machine - II Non-linearly separable data	R1
During CH
Post CH

Session 14

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	14	Genetic Algorithms - I Example, properties, How to solve a problem?, Operator Selection Methods, Basic Genetic Algorithm Operators	R2 & R3
During CH
Post CH

Session 15

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	15	Genetic Algorithms - II Representing Hypotheses, GABIL, Hypothesis Search Space, Population Evolution, Schema theorem	R2 & R3
During CH
Post CH

Session 16

Type	Content Ref.	Topic Title	Study/HW Resource Reference
Pre CH	9 - 15	Review of Session 9 to 15	Books, Web references and Slides (L9-L15)
During CH
Post CH

 Evaluation Scheme:  

Legend: EC = Evaluation Component; AN = After Noon Session; FN = Fore Noon Session

No	Name	Type	Duration	Weight	Day, Date, Session, Time
EC-1	Quiz-I/ Assignment-I	Online	-	5%	February 1 to 10, 2018
	Quiz-II	Online	-	5%	March 1 to 10, 2018
	Assignment	Online	-	10%	March 20 to 30, 2018
EC-2	Mid-Semester Test	Closed Book	2 hours	30%	04/03/2018 (FN) 10 AM – 12 Noon
EC-3	Comprehensive Exam	Open Book	3 hours	50%	22/04/2018 (FN) 9 AM – 12 Noon

Syllabus for Mid-Semester Test (Closed Book): Topics in Session Nos.  1 TO 8

Syllabus for Comprehensive Exam (Open Book): All topics (Session Nos. 1 to 16)

Important links and information:

Elearn portal: https://elearn.bits-pilani.ac.in

Students are expected to visit the Elearn portal on a regular basis and stay up to date with the latest announcements and deadlines.

Contact sessions: Students should attend the online lectures as per the schedule provided on the Elearn portal.

Evaluation Guidelines:

1.       EC-1 consists of either two Assignments or three Quizzes. Students will attempt them through the course pages on the Elearn portal. Announcements will be made on the portal, in a timely manner.

2.       For Closed Book tests: No books or reference material of any kind will be permitted.

3.       For Open Book exams: Use of books and any printed / written reference material (filed or bound) is permitted. However, loose sheets of paper will not be allowed. Use of calculators is permitted in all exams. Laptops/Mobiles of any kind are not allowed. Exchange of any material is not allowed.

4.       If a student is unable to appear for the Regular Test/Exam due to genuine exigencies, the student should follow the procedure to apply for the Make-Up Test/Exam which will be made available on the Elearn portal. The Make-Up Test/Exam will be conducted only at selected exam centres on the dates to be announced later.

It shall be the responsibility of the individual student to be regular in maintaining the self study schedule as given in the course handout, attend the online lectures, and take all the prescribed evaluation components such as Assignment/Quiz, Mid-Semester Test and Comprehensive Exam according to the evaluation scheme provided in the handout.