Deakin University
Browse

File(s) not publicly available

Evolving deep architecture generation with residual connections for image classification using particle swarm optimization

journal contribution
posted on 2021-11-28, 00:00 authored by T Lawrence, L Zhang, K Rogage, Chee Peng LimChee Peng Lim
Automated deep neural architecture generation has gained increasing attention. However, exiting studies either optimize important design choices, without taking advantage of modern strategies such as residual/dense connections, or they optimize residual/dense networks but reduce search space by eliminating fine-grained network setting choices. To address the aforementioned weaknesses, we propose a novel particle swarm optimization (PSO)-based deep architecture generation algorithm, to devise deep networks with residual connections, whilst performing a thorough search which optimizes important design choices. A PSO variant is proposed which incorporates a new encoding scheme and a new search mechanism guided by non-uniformly randomly selected neighboring and global promising solutions for the search of optimal architectures. Specifically, the proposed encoding scheme is able to describe convolutional neural network architecture configurations with residual connections. Evaluated using benchmark datasets, the proposed model outperforms existing state-of-the-art methods for architecture generation. Owing to the guidance of diverse non-uniformly selected neighboring promising solutions in combination with the swarm leader at fine-grained and global levels, the proposed model produces a rich assortment of residual architectures with great diversity. Our devised networks show better capabilities in tackling vanishing gradients with up to 4.34% improvement of mean accuracy in comparison with those of existing studies.

History

Journal

Sensors

Volume

21

Issue

23

Article number

ARTN 7936

Pagination

1 - 23

Publisher

MDPI / MDPI AG (Multidisciplinary Digital Publishing Institute)

Location

Basel, Switzerland

ISSN

1424-8220

eISSN

1424-8220

Language

English

Publication classification

C1 Refereed article in a scholarly journal