More than 3,000 box tree caterpillars – which are originally from east Asia – were recorded last year, according to the Royal Horticultural Society (RHS).
This is a sharp rise from the 800 sighted between 2015 and 2016.
The RHS said the caterpillars can “completely defoliate box plants”, a type of shrub commonly used for hedges.
The destructive insects were first sighted in Britain 10 years ago and have become widespread in the home counties.
Butterfly Conservation said the caterpillars have been brought over by the horticultural trade and climate change.
In 2017 the insects were named the top pest by the RHS in their annual top ten plant pests and diseases rankings.
Slugs and snails ranked at number four on the list.
The caterpillars, which turn into moths, have yellow, black and white stripes on their bodies.
The moths have white wings with a brown border or can be completely brown.