File(s) not publicly available
Evolving deep architecture generation with residual connections for image classification using particle swarm optimization
journal contribution
posted on 2021-11-28, 00:00 authored by T Lawrence, L Zhang, K Rogage, Chee Peng LimChee Peng LimAutomated deep neural architecture generation has gained increasing attention. However, exiting studies either optimize important design choices, without taking advantage of modern strategies such as residual/dense connections, or they optimize residual/dense networks but reduce search space by eliminating fine-grained network setting choices. To address the aforementioned weaknesses, we propose a novel particle swarm optimization (PSO)-based deep architecture generation algorithm, to devise deep networks with residual connections, whilst performing a thorough search which optimizes important design choices. A PSO variant is proposed which incorporates a new encoding scheme and a new search mechanism guided by non-uniformly randomly selected neighboring and global promising solutions for the search of optimal architectures. Specifically, the proposed encoding scheme is able to describe convolutional neural network architecture configurations with residual connections. Evaluated using benchmark datasets, the proposed model outperforms existing state-of-the-art methods for architecture generation. Owing to the guidance of diverse non-uniformly selected neighboring promising solutions in combination with the swarm leader at fine-grained and global levels, the proposed model produces a rich assortment of residual architectures with great diversity. Our devised networks show better capabilities in tackling vanishing gradients with up to 4.34% improvement of mean accuracy in comparison with those of existing studies.
History
Journal
SensorsVolume
21Issue
23Article number
ARTN 7936Pagination
1 - 23Publisher
MDPI / MDPI AG (Multidisciplinary Digital Publishing Institute)Location
Basel, SwitzerlandPublisher DOI
Link to full text
ISSN
1424-8220eISSN
1424-8220Language
EnglishPublication classification
C1 Refereed article in a scholarly journalUsage metrics
Categories
Keywords
ALGORITHMChemistryChemistry, Analyticaldeep architecture generationdeep residual networkEngineeringEngineering, Electrical & ElectronicGRADIENTimage classificationInstruments & InstrumentationNEURAL-NETWORKSparticle swarm optimizationPhysical SciencesScience & TechnologyTechnologyDistributed ComputingEcology
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC