CSE424: Neural Network

/in /by

Syllabus

WeeksTopicsLecturesPresentation Topics
Week-1Neural Network Basics, Multilayer Perceptron, Linear Classifiers, Loss calculation, Log likelihood loss, Cross Entropy Loss, Softmax Classifier, Different Activation Functions and their Derivatives2
Week-2Gradient Descent, Chain Rule for Derivatives, Back Propagation, Update Rule, Implementation of Multilayer Perceptron from Scratch that uses back propagation2
Week-3Convolutional Neural Network, Filters, Kernels, Convolutional Layer, Max Pool Layer, Activation Function ReLU, Batch Normalization, Implementation of CNN from Scratch2
Week-4Capacity, Overfitting, Under fitting, Regularization, Weight Decay, Dropout, Batch Normalization, Convolutional AutoEncoder, Semantic Segmentation, Different up-sampling method (Deconvolution, Reverse Maxpool)2Presentation: Semantic Segmentation Presentation
1. Segnet
2. FCN-8
Week-5Attention, Where CNN pays attention for classification Concept:
Class Activation Map (CAM)		21. GradCAM
Learn to Pay Attention
Week-6Object Detection, Object localization , Region Proposal, Regional Convolutional Neural Network (R-CNN) , Mask R-CNN21. YOLO
2.Fast R-CNN
3.Faster R-CNN
Week-7Word Embedding, Word2vec, Negative Sampling, Character Level Embedding, Sentence Level Embedding21. Attention all you need
2. BERT
Week-8LSTM/GRU for language model, Neural Machine Translation, LST/GRU + Attention, Image Captioning21. Show, Attend, and Tell
Week-9Self-Attention, Transformer for Neural Machine Translation21. Transformer-XL
Week-10Introduction to Graph Embedding, Node2vec, Graph Convolution Network21. Representation Learning on Graphs: Method and Application
Week-11Graph Neural Network (GNN) style Embedding, Graph Attention Network (GAT) style embedding21. GraphSage
Week-12Advanced Topics Variational Auto Encoder, Generative Adversarial Network, Few/Zero Shot Learning2

CEN/CSE 421: Machine Learning

/in /by

Course Objective

We have entered the era of big data. This deluge of data calls for automated methods of data analysis, which is what machine learning or pattern recognition provides. Machine learning as a set of methods that can automatically detect patterns in data, and then use the uncovered patterns to predict future data, or to perform other kinds of decision making under uncertainty (such as planning how to collect more data!). This introductory pattern recognition/machine learning course will give an overview of many popular models and algorithms used in modern machine learning – both statistical and non-statistical. The course will give the student the basic ideas and intuition behind these methods, as well as a more formal understanding of how and why they work. Students will have an opportunity to experiment with machine learning techniques and apply them on real life problems.

Course Information

Syllabus

# of LecturesTopics
1Machine Learning/Pattern Recognition basics:
Supervised and Unsupervised Learning – Classification, Clustering and Regression, Parametric and Non-parametric Models, Curse of Dimensionality, Over-fitting, and Model Selection, Performance Measures
1Data:  Attribute types, Basic Statistical description of Data, Review of probability theory
2Bayesian Decision Theory: Likelihood Ratio Test, Bayes Risk; Bayes, ML and MAP Criteria, Naive Bayes classifier
1Normal Variables and its Discriminant Analysis
1Parametric Density Estimation: MLE, Bayesian Density Estimation
1Nonparametric Density Estimation: Kernel Density Estimators and Nearest Neighbor Method
1Regression: Linear Regression Analysis and Bayesian Linear Regression
2Decision Trees and Random Forests, Ensemble Methods: Bagging and Boosting
4Feature Selection and Extraction, Dimensionality Reduction : PCA and SVD
3Linear Models for Classification: Fisher’s Linear Discriminant, Support Vector Machines
2Introduction to Graphical Models: Bayesian Networks, Exact and Approximate Inference Methods
2*Introduction to Reinforcement Learning: Policy gradient, Q-learning

CSE417: Data mining and Data warehouse

/in /by

Course Description

We will learn theory, concepts, and applications on how to extract useful information from huge amounts of data.

Course Information

Syllabus

WeekSessionTopicsResourcesAssignments
Week-1Session-1Data Matrix, Attributes, Vector Recap, Basic Statistics, Distributions, PDF, CDFhttps://dataminingbook.info/resources/
Book: DMML, Chapter 1 and Chapter 2		Assignment-1
Week-2Session-2Multivariate Gaussian, Covariance Matrix, Geometry of the multivariate normal, Diagonalization of Covariance Matrixhttps://dataminingbook.info/resources/
Book: DMML, Chapter 1 and Chapter 2
Week-3Session-3Frequent Itemset Mining, The Market-Basket Model, Mining Association Rules, Finding Frequent Pairs, A-Priori Algorithm, FP Growth, *Eclat algorithmhttp://www.mmds.org/#book
Book: MMDS, Chapter 6		Assignment-2
Week-4Session-4Mining Data Streams, General Stream Processing Model, Sampling from a Data Stream, *Queries over a (long) Sliding Windowhttp://www.mmds.org/#book
Book: MMDS, Chapter 4
Week-5Session-5Analysis of Large Graphs: Link Analysis, PageRank, Topic Specific Page rank, *Sim Rankhttp://www.mmds.org/#book
Book: MMDS, Chapter 5		Assignment-3
Week-6Session-6Recommender Systems, Content-based Systems, Collaborative Filteringhttp://www.mmds.org/#book/>Book: MMDS, Chapter 9
Week-7Session-7Recommender Systems, Latent Factor Models, SVDhttp://www.mmds.org/#book
Book: MMDS, Chapter 9, 11		Assignment-4
Week-8Session-8Application of SVD in recommender system, *SVD for dimension reductionhttp://www.mmds.org/#book
Book: MMDS, Chapter 9, 11
Week-9Session-9Analysis of Large Graphs: Community Detection, Betweenness, Modularity, Graph Partitioning, *Graph Cut, Spectral Partitioninghttp://www.mmds.org/#book
Book: MMDS, Chapter 10		Assignment-5
Week-10Session-10Map-Reduce and the New Software Stackhttp://www.mmds.org/#book
Book: MMDS, Chapter 10		Assignment-6
Week-11Session-11*Finding Similar Items: Locality Sensitive Hashing, *Distance Measure, *MinHashinghttp://www.mmds.org/#book
Book: MMDS, Chapter 3
Week-12Session-12TBATBA

CSE317: Numerical Methods

/in /by

Course Objective

The objective of this course is to introduce the student computational methods required by engineers, mathematicians, physicists and economists to explore complex systems. Mathematical models developed to explore complex systems can be rarely “solvable” algebraically and hence computational methods have been developed. This course introduces such methods that range from techniques for system of linear equations, nonlinear equations, approximation of functions, interpolation, clustering, least square data fitting and classification, differentiation and integration. More emphasis will be put on applied linear algebra topics which are prerequisite for Artificial Intelligence, Machine Learning, and other advanced courses. We will make use of Matlab programming to implement and analyze the methods.

Course Information

Syllabus

TopicsReadings# of lectures
Approximation errors and approximating single variable functions
Floating point number system and error in number representation, review of derivatives, Taylor Series, finding optima of single variable functionsCh 3, 4 NME1
Finding roots of single variable functions – Bisection, Secant and Newton-Raphson MethodCh 5, 6 NME1
Vectors and Matrices
Vectors – review of vector notation, vector operations, linear and affine multivariable functions, complex vectors, complexity of vector computations,
applications: vector representation of data (e.g., images, documents, timeseries, features), vector representation of linear and affine functions (e.g., regression, Linear (Taylor) approximation of multivariable function functions)		Ch 1,2 VMLS2
Norms and distances – Euclidean norm and distances, properties (Cauchy-Schwarz and triangle inequalities, Pythagorean theorem), statistical measurements of data: average, rms, standard deviation, and angle between vectors and correlation, covariance; representation of hyperplanes,
application: single variable linear regression, k-means clustering		Ch 3,4 VMLS2
Direct Methods for Solving System of Linear Equations
Solving system of linear equations using LU decomposition,  application: Polynomial interpolation and Vandermonde matrix, applications of solving system of linear equationsCh 8 VMLS2
Matrix Inverses: Left and right inverses, solving system of linear equations using matrix inverses, Gram matrix and Pseudo-inverseCh 5, 11 VMLS2
Orthogonality and Least Square Methods
Basis, orthogonality and inner products: basis and change of basis, Orthogonal basis, Gram-Schmidt, modified-Gram Schmidt algorithms, QR decomposition of matrices, *Householder reflections,
application: solving system of linear equations using QR factorization, *lower dimensional data representation		Ch 5, 10, 11 VMLS2
Linear least-Squares: solution to over-determined systems, normal equation and pseudo inverse of a matrix, computing pseudo inverse using QR and Cholesky factorization, solving least squares using matrix-vector derivates,
application: data fitting and least-square regression, feature engineering, Least-square classification, regularized least square data fitting, *least square function approximation		Ch 12-14 VMLS; Ch 17 NME3
*Interpolation
Interpolation using monomial and Lagrange bases will be discussed in Linear equation lecture. *Interpolation using other basis functions: Newton, Legendre, Chebyshev bases, Hermite interpolation, cubic spline interpolationCh 18 NME2
Numerical Differentiation and Integration
Finite divided difference approximation of derivatives, Trapezoidal rule, Simpson’s ruleCh 22, 23 NME1
Problem Condition, Algorithm StabilityCh 6, 7 (notes)2

UFlow-Net: A Unified Approach for Improved Video Frame Interpolation

/in /by

In computer vision, video frame interpolation  plays a significant role in video enhancement by synthesizing intermediate frames to improve temporal resolution and visual quality. This techniques help reduce motion blur, create smoother slow motion videos and enhancing total viewing experience, especially in low frame rate video. This is vital for application like video processing, streaming and video restoration. We are developing UFlow-Net, a deep learning-based model that improves frame interpolation accuracy.

The process starts with a dataset of three consecutive video frames. The first and third frames are used as inputs, while the second frame is used as a reference for evaluation. These frames go through preprocessing, such as resizing, normalizing, and stacking the frames.

Next, the preprocessed frames are passed into UFlow-Net, which consists of two key steps. The Flow-Enhanced Encoder-Decoder captures motion and spatial details from the input frames, and reconstructs the features, keeping the motion consistent. The Refined Frame Synthesis step, refines the features more and generates the missing middle frame by using the learned motion patterns and spatial relationships. We evaluate our model using PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index Measure). Our Model achieved a PSNR of 35.65 dB and SSIM score 0.97. 

Relevant publications: 

F. Israq, S. B. Alam, H. Khatun, S.S. Sarker, S.T. Bhuiyan, M. Haque, R. Rahman and S. Kobashi ” UFlow-Net: A Unified Approach for Improved Video Frame Interpolation” in Proc. 2024 27th International Conference on Computer and Information Technology (ICCIT), Cox’s Bazar, Bangladesh, Dec. 20-22, 2024. 

ROI-Guided Lumbar Spine Degeneration Detection

/in /by

Lumbar Spine Degeneration, otherwise known as Disk Degeneration Disease, is the deterioration/weakening of the intervertebral disks in the lower back, causing them to lose their ability to absorb shock and potentially leading to pain and discomfort due to nerve compression. While lumbar degeneration is a natural part of aging, its progression and severity can vary from person to person. Early detection and classification of the type and stage of degeneration are crucial for effective management and treatment. We are using the power of AI to automate the detection of lumbar degeneration.

Firstly, We merge and pre-process the Magnetic Resonance Imaging (MRI) scans to convert them to RGB images, comprised with multi-label diseases per image.Then, we used an YOLO(You Only Look Once) architecture to detect the region of interests (ROI) that include the intervertebral disks in the MRI scans from both Sagittal and Axial Plane. Finally, the ROIs are classified into different degree of degeneration.

The goal of this research is to detect Lumbar Spine Degeneration as early as possible in order to ensure the best quality of life and outcome for the patient.

Detection-Guided Kidney Abnormality Segmentation

/in /by

Tumors and cysts are two major kidney abnormalities which can lead to cancer if left undetected and untreated timely. The current diagnostic process depend on computed tomography (CT) scan screening which is time-consuming and specialist-dependent. This leads to fatigue for radiologists and doctors. As a result, diagnostic errors increase. To enhance the diagnostic process, we are developing an AI-based automated method for kidney abnormality detection. 

The first step in this method is kidney detection using the YOLOv8 model. This is conducted on 2D sliced images extracted from 3D CT. The detected kidney regions are then cropped to reduce the background area. After that, abnormal region segmentation is conducted using the U-Net segmentation model. The abnormal region consists of either cyst or tumor or both. This produces a mask of abnormal region for each 2D slice. The 2D mask slices are combined to construct a 3D mask of the abnormal region. 

This study aims to automate the detection process of kidney tumor and cyst, offering a faster and enhanced approach to assist the doctors and radiologists in diagnostic process.

Relevant publications:

1. J. Faruk, S. B. Alam, S. S. T. Elma, S. Wasi, R. Rahman and S. Kobashi, “Kidney Abnormality Detection Using Segmentation-Guided Classification on Computed Tomography Images,” 2024 International Conference on Machine Learning and Cybernetics (ICMLC), Miyazaki, Japan, 2024, pp. 414-419, doi: 10.1109/ICMLC63072.2024.10935113.

2. S. Wasi, S. B. Alam, R. Rahman, M. A. Amin and S. Kobashi, “Kidney Tumor Recognition from Abdominal CT Images using Transfer Learning,” 2023 IEEE 53rd International Symposium on Multiple Valued Logic (ISMVL), Matsue, Japan, 2023, pp. 54-58, doi: 10.1109/ISMVL57333.2023.00021