1. (Jan 26) Course page is online.
  2. (Jan 26) Syllabus is available.
  3. (Jan 30) Slides for Introduction to Pattern Recognition are available.
  4. (Feb 1) Slides for Bayesian Decision Theory are available.
  5. (Feb 13) First part of the slides for Parametric Models is available.
  6. (Feb 20) Second part of the slides for Parametric Models is available.
  7. (Feb 22) Homework assignment 1 is available.
  8. (Feb 27) Third part of the slides for Parametric Models is available.
  9. (Feb 27) There will be no class on March 1, 2007.
  10. (Mar 3) Deadlines for project are added.
  11. (Mar 6) Fourth part of the slides for Parametric Models is available.
  12. (Mar 13) Slides for Non-parametric Methods are available.
  13. (Mar 20) Slides for Feature Reduction and Selection are available.
  14. (Mar 21) Homework assignment 2 is available.
  15. (Mar 27) First part of the slides for Non-Bayesian Classifiers is available.
  16. (Mar 27) Slides for Feature Reduction and Selection are updated with more examples.
  17. (Mar 29) Second part of the slides for Non-Bayesian Classifiers is available.
  18. (Apr 4) Third part of the slides for Non-Bayesian Classifiers is available.
  19. (Apr 17) Slides for Unsupervised Learning and Clustering are available.
  20. (Apr 26) Slides for Algorithm-Independent Learning Issues are available.
  21. (May 3) Homework assignment 3 is available.
  22. (May 8) Slides for Structural and Syntactic Pattern Recognition are available.
  23. (May 10) Homework assignment 3 due date is postponed to May 20, 2007.
  24. (May 21) Project presentations will be made at EA 502 during 13:30-15:30 on May 24th.


Instructor: Selim Aksoy
Office: EA 423

Course Information

Schedule: Tue 8:40-10:30, Thu 9:40-10:30 (EA 502)
Office hours: Fri 10:40-12:00
Mailing list:
Prerequisites: Probability theory, statistics, linear algebra


Lecture Schedule



Introduction to Pattern Recognition

[ Slides ]

(Jan 30, Feb 1)

  • Pattern recognition systems
  • The design cycle
  • An example
  • DHS Ch 1, Appendix A.1-A.2, A.4-A.5

Bayesian Decision Theory

[ Slides ]

(Feb 6, 8, 13)

  • Modeling using continuous and discrete features
  • Discriminant functions
  • The Gaussian density
  • Error estimation
  • DHS Ch 2.1-2.9

Parametric Models

[ Slides: Part 1 | Part 2 | Part 3 | Part 4 ]

(Feb 15, 20, 22, 27, Mar 6, 8, 13)

  • Maximum-likelihood estimation
  • Bayesian estimation
  • Expectation-Maximization and mixture density estimation
  • Hidden Markov Models
  • Bayesian Belief Networks
  • DHS Ch 3.1-3.5, 3.9, 10.2-10.4, 3.10, 2.11

Non-parametric Methods

[ Slides ]

(Mar 15, 20)

  • Density estimation
  • Histogram-based estimation
  • Parzen windows estimation
  • Nearest neighbor estimation
  • DHS Ch 4.1-4.4

Feature Reduction and Selection

[ Slides ]

(Mar 22)

  • Problems of dimensionality
  • Component analysis
    • Principal components analysis (PCA)
    • Linear discriminant analysis (LDA)
  • Feature selection
  • DHS Ch 3.7-3.8, 10.13-10.14

Non-Bayesian Classifiers

[ Slides: Part 1 | Part 2 | Part 3 ]

(Mar 27, 29, Apr 3, 5)

  • k-nearest neighbor classifier
  • Linear discriminant functions
  • Support vector machines
  • Neural networks
  • Decision trees
  • DHS Ch 4.5-4.6, 5.1-5.3, 5.11, 6.1-6.3, 8.1-8.3

Spring Break

(Apr 9-13)

No class

Unsupervised Learning and Clustering

[ Slides ]

(Apr 17, 19, 24)

  • Criterion functions for clustering
  • k-means clustering
  • Hierarchical clustering
  • Graph-theoretic clustering
  • Cluster validity
  • DHS Ch 10.1, 10.6-10.7, 10.9-10.10, 10.12

Algorithm-Independent Learning Issues

[ Slides ]

(Apr 26, May 1)

  • No Free Lunch Theorem
  • Resampling for classifier design
  • Comparing classifiers
  • Combining classifiers
  • DHS Ch 9.1-9.2, 9.5-9.7

Structural and Syntactic Pattern Recognition

[ Slides ]

(May 3, 8, 10)

  • Recognition with strings
  • Grammatical methods
  • Graph-theoretic methods
  • DHS Ch 8.5-8.6


  1. Homework assignment 1 (Due: March 8, 2007 as hardcopy in the class)
  2. Homework assignment 2 (Due: April 6, 2007 as online submission)
  3. Homework assignment 3 (Due: May 20, 2007 as online submission)

Late submission policy: Unless you make prior arrangements with me (before the due date), no late homework will be accepted.


The purpose of the project is to enable the students to get some hands-on experience in the design, implementation and evaluation of pattern recognition algorithms by applying them to real-world problems. The objective is to try multiple algorithms for different steps of the design cycle such as feature extraction and selection, model learning and estimation, classification and evaluation, to get as high an accuracy as possible on the selected datasets.

You can use your own data from your thesis research, select datasets from the list of data resources below, or contact the instructor for data from ongoing research on multimedia analysis (including image, video, audio and text data) or image classification. In any case, you should get prior approval before starting your project.

You are free to use any programming language but Matlab is strongly recommended because it is very convenient for prototyping and has many tools available for pattern recognition. You can write the codes yourself or use any code that is available in the public domain. In case you use somebody else's code, you are required to properly cite its source and know the details of the algorithms that the code implements.

You are required to work as a group, and submit a project proposal, an interim progress report, a final report written in a conference paper format, and make a presentation during the finals week. Tentative schedule of the project is as follows:

All reports and software can be submitted using the online form.

Final report guidelines:

# Projects Members
1 Recognition of TCP traffic anomalies Caglar Arslan, Osman Pamuk
2 Writer identification using handwritten text H. Hakan Ari, Barkin Basarankut
3 Authorship attribution Ilker Nadi Bozkurt, Erkan Uyar, Ozgur Baglioglu
4 Hand-gesture recognition Muzaffer Akbay, Kardelen Hatun, Omer Sezgin Ugurlu
5 Cost sensitive feature selection and classification Mumin Cebe, Cihan Ozturk
6 Building recognition from aerial images using Bayesian belief networks Melih Kandemir, Alexander Suhre, A. Burak Tosun
7 Landmark identification and detection for SLAM algorithm in UAVs Akin Avci, Mehmet Kok, S. Tuncer Erdogan
8 Automatic summary extraction from text documents using support vector machines Nagehan Pala
9 Motion annotation Selen Pehlivan
10 License plate detection on images Suleyman Kardas
11 Classification of characters in Ottoman script Berk Berker, Emine Busra Celikkaya, Ismet Zeki Yalniz

Grading Policy

Homework and quiz:55%
Term project:40%
Class participation:5%

Related Links