Iterative Facial Inpainting Using Reverse Generator

In this paper, we propose an efficient solution to the facial image painting problem using the Cyclic Reverse Generator (CRG) architecture, which provides an encoder-generator model. We use the encoder to embed a given image to the generator space and incrementally inpaint the masked regions until a plausible image is generated; a discriminator network is utilized to assess the generated images during the iterations. We empirically observed that only a few iterations are sufficient to generate realistic images with the proposed model. 

Y Dogan, HY Keles – Neural Computing and Applications journal, 2022

[Paper] [Arxiv]

Using Motion History Images with 3D Convolutional Networks in Isolated Sign Language Recognition

In this paper, we propose an isolated sign language recognition model based on a model trained using Motion History Images (MHI) that are generated from RGB video frames. RGB-MHI images represent spatio-temporal summary of each sign video effectively in a single RGB image. We propose two different approaches using this RGB-MHI model. In the first approach we use the RGB-MHI model as a motion-based spatio-temporal attention module integrated into a 3D-CNN architecture. In the second approach, we use RGB-MHI model features directly with the features of a 3D-CNN model using a late fusion technique.

OM Sincan, HY Keles – IEEE Access, 2022

[Paper] [Arxiv]

A Hierarchical Approach to Remote Sensing Scene Classification

Remote sensing scene classification deals with the problem of classifying land use/cover of a region from images. To predict the development and socioeconomic structures of cities, the status of land use in regions are tracked by the national mapping agencies of countries. Many of these agencies use land use types that are arranged in multiple levels. In this paper, we examined the efficiency of a hierarchically designed CNN based framework that is suitable for such arrangements. We use NWPU-RESISC45 dataset for our experiments and arranged this data set in a two level nested hierarchy. We have two cascaded deep CNN models initiated using DenseNet-121 architectures.

O Sen, HY Keles – PFG Journal, 2022

[Paper] [Arxiv]

Evaluation of Hidden Markov Models Using Deep CNN Features in Isolated Sign Recognition

In this study, we provide a framework that is composed of three modules to solve isolated sign recognition problem using different sequence models. The dimensions of deep features are usually too large to work with HMM models. To solve this problem, we propose two alternative CNN based architectures as the second module in our framework, to reduce deep feature dimensions effectively.

A Tur, HY Keles – Multimedia Tools and Applications, 2021

[Paper] [Arxiv]

Semi-supervised image attribute editing using generative adversarial networks  

Image attribute editing is a challenging problem that has been recently studied by many researchers using generative networks. The challenge is in the manipulation of selected attributes of images while preserving the other details. The method to achieve this goal is to find an accurate latent vector representation of an image and a direction corresponding to the attribute. Almost all the works in the literature use labeled datasets in a supervised setting for this purpose. In this study, we introduce an architecture called Cyclic Reverse Generator (CRG), which allows learning the inverse function of the generator accurately via an encoder in an unsupervised setting by utilizing cyclic cost minimization. 

Y Dogan, HY Keles - Neurocomputing, 2020

[Paper] [Arxiv]

Towards Disease-aware Image Editing of Chest X-rays 

Disease-aware image editing by means of generative adversarial networks (GANs) constitutes a promising avenue for advancing the use of AI in the healthcare sector. Here, we present a proof of concept of this idea. While GAN-based techniques have been successful in generating and manipulating natural images, their application to the medical domain, however, is still in its infancy. Working with the CheXpert data set, we show that StyleGAN can be trained to generate realistic chest X-rays. Inspired by the Cyclic Reverse Generator (CRG) framework, we train an encoder that allows for faithfully inverting the generator on synthetic X-rays and provides organ-level reconstructions of real ones. 

A Saboo, SN Ramachandran, K Dierkes, HY Keles - MED-NEURIPS 2020

[Paper] [Presentation]

AUTSL: A Large Scale Multi-modal Turkish Sign Language Dataset and Baseline Methods

In this study, we present a new large-scale multi-modal Turkish Sign Language dataset (AUTSL) with a benchmark and provide baseline models for performance evaluations. Our dataset consists of 226 signs performed by 43 different signers and 38,336 isolated sign video samples in total. Samples contain a wide variety of backgrounds recorded in indoor and outdoor environments. Moreover, spatial positions and the postures of signers also vary in the recordings. Each sample is recorded with Microsoft Kinect v2 and contains color image (RGB), depth and skeleton data modalities. We prepared benchmark training and test sets for user independent assessments of the models. We trained several deep learning based models and provide empirical evaluations using the benchmark; we used Convolutional Neural Networks (CNNs) to extract features, unidirectional and bidirectional Long Short-Term Memory (LSTM) models to characterize temporal information.

OM Sincan, HY Keles – IEEE Access, 2020

[Paper] [Arxiv][AUTSL Dataset]

Learning Multi-scale Features for Foreground Segmentation  

Foreground segmentation algorithms aim at segmenting moving objects from the background in a robust way under various challenging scenarios. Encoder–decoder-type deep neural networks that are used in this domain recently perform impressive segmentation results. In this work, we propose a variation of our formerly proposed method that can be trained end-to-end using only a few training examples.

LA Lim, HY Keles – Pattern Analysis and Applications, 2020

[Paper] [Arxiv] [Code]

Hand and Face Segmentation with Deep Convolutional Networks Using Limited Labelled Data

In this paper, we propose two segmentation networks that mark face and hands from static images for sign language recognition using only a few training data. Our networks have encoder-decoder structure that contains convolutional, max pooling and upsampling layers; the first one is a U-Net based network and the second one is a VGG-based network.

OM Sincan, S Gencoglu, M Bacak, HY Keles –  IEEE ISMSIT, 2019

 [Paper] [Code]

Isolated Sign Recognition with a Siamese Neural Network of RGB and Depth Streams

Sign recognition is a challenging problem due to high variance of the signs among different signers and multiple modalities of the input information. In this work, we propose a Siamese Neural Network (SNN) architecture that is used to extract features from the RGB and the depth streams of a sign frame in parallel.

AO Tur, HY Keles – IEEE EUROCON, 2019

 [Paper]

DeepVoCoder: A CNN model for compression and coding of narrow band speech

This paper proposes a convolutional neural network (CNN)-based encoder model to compress and code speech signal directly from raw input speech. Although the model can synthesize wideband speech by implicit bandwidth extension, narrowband is preferred for IP telephony and telecommunications purposes.

HY Keles, J Rozhon, HG Ilk, M Voznak – IEEE Access, 2019

 [Paper]

Scene Recognition with Deep Learning Methods Using Aerial Images

In this paper, two novel deep learning architectures are proposed to solve the scene classification problem using aerial images. In model evaluations, we used one of the largest open access dataset, i.e. NWPU-RESIS45 dataset, that contains 45 different categories and in total 31500 samples. 95.7% accuracy that we get with the developed models show a competitive performance with the state-of-the-art methods.

O Sen, HY Keles – IEEE SIU,  2019

 [Paper]

Stability and diversity in generative adversarial networks

Generative Adversarial Networks (GANs) enable generating photo-realistic images more successfully compared to other generative models. In this study, we empirically examined the state-of-the-art cost functions, regularization techniques and network architectures that have recently been proposed to deal with these problems, using CelebA dataset.

Y Dogan, HY Keles – IEEE SIU, 2019

[Paper]

Isolated Sign Language Recognition with Multi-scale Features using LSTM

Sign language recognition systems are used to convert signs in video streams to text automatically. In this work, an original isolated sign language recognition model is created using Convolutional Neural Networks (CNNs), Feature Pooling Module and Long Short-Term Memory Networks (LSTMs).

OM Sincan, AO Tur, HY Keles – IEEE SIU, 2019

 [Paper]

Foreground segmentation using convolutional neural networks for multiscale feature encoding

We propose two robust encoder-decoder type neural networks that generate multi-scale feature encodings in different ways and can be trained end-to-end using only a few training samples. Using the same encoder-decoder configurations, in the first model, a triplet of encoders take the inputs in three scales to embed an image in a multi-scale feature space.

LA Lim, HY Keles – Pattern Recognition Letters, 2018

 [Paper][Arxiv] [Code]

A Comparative Study of HMMs and LSTMs on Action Classification with Limited Training Data

Action classification from video streams is a challenging problem, especially when there is a limited number of training data for different actions. In this work, we examined the performances of Hidden Markov Models (HMM) and long short-term memory (LSTM) based recurrent neural network models using the same sequence classification framework with the well known KTH action dataset.

EC Alp, HY Keles – Advances in Intelligent Systems and Computing Book, 2018

[Paper]

Embedding Parts in Shape Grammars using a Parallel Particle Swarm Optimization Method on GPUs

Embedding emergent parts in shape grammars is computationally challenging. The first challenge is the representation of shapes, which needs to enable reinterpretation of parts regardless of the creation history of the shapes. The second challenge is the relevant part searching algorithm that provides an extensive exploration of the design space–time efficiently. In this work, we propose a novel method to solve both problems.

HY Keles – AI EDAM, 2018

 [Paper]

Moving object detection and classification in surveillance systems using moving cameras

In this paper, we present a novel method to detect and classify moving objects from surveillance videos that are obtained from a moving camera. In our method, we first estimate the camera motion by interpreting the movement of interest points in the scene. Then, we eliminate the camera motion and find candidate regions that belong to the moving objects.

OM Sincan, HY Keles, S Tosun – Communications Faculty of Sciences University of Ankara Series A2-A3 Physical Sciences and Engineering, 2018

 [Paper]

Action recognition using MHI based Hu moments with HMMs

Action recognition from video streams is among the active research topics in computer vision. The challenge is on the identification of the actions robustly regardless of the variations imposed by appearances of actions performed by different people. This paper proposes a Hidden Markov Model (HMM) based approach to model actions using Hu moments that are computed using a modified Motion History Images (MHI).

EC Alp, HY Keles – IEEE EUROCON , 2017

 [Paper]

Person identification using functional near-infrared spectroscopy signals using a fully connected deep neural network

In this study, we investigate the suitability of functional near-infrared spectroscopy signals (fNIRS) for person identification using data visualization and machine learning algorithms. We first applied two linear dimension reduction algorithms: Principle Component Analysis (PCA) and Singular Value Decomposition (SVD) in order to reduce the dimensionality of the fNIRS data.

OM Sincan, HY Keles, Y Kır, A KUSMAN, B Baskak – Communications Faculty of Sciences University of Ankara Series A2-A3 Physical Sciences and Engineering, 2017

 [Paper]