For clarity, 'public holidays' and 'bank holidays', which tend to be used interchangeably, are two different things. The former are 'common law holidays', which have been observed through custom and practice (e.g. Good Friday, Christmas Day) whereas the latter are days fixed by statute and relate to when financial institutions (banks etc) are closed. For England and Wales, the term 'public holiday' tends to cover both bank holidays and the common law variety, so for this article, we shall use that term.
The starting point is to consider the basic holiday entitlement under the Working Time Regulations 1988 (WTR). This entitles workers to at least 5.6 weeks' paid holiday in each leave year (pro rata for part-time). Any public holidays given by employers can count towards the statutory minimum, but workers do not have an automatic (statutory) right to time off on specific public holidays.
What the worker is entitled to should be governed by their written terms of employment. It is quite common for holiday entitlement to be stated as comprising [X] number of days plus the normal public/bank holidays, or sometimes being a little more precise and referring to the normal 8 public/bank holidays. In that situation, there would be no automatic entitlement to the additional (9th) public holiday on 3 June and it would be down to the employer to decide (based on business needs) whether to allow workers to have that day off; and, perhaps, if some could and some couldn't, to give another day in lieu, or perhaps pay an overtime rate for those who miss out.
If the contract states that workers are entitled to [X] days paid holiday, in addition to all public holidays, then clearly the extra 9th public holiday on 3 June would be automatically caught. Equally, if the contract states that the holiday entitlement is inclusive of public holidays, then whilst 3 June would fall into that, it would not give the worker an extra day off, as it falls within their total 'inclusive' annual holiday entitlement.
If there are no written terms of employment (itself a breach of employment law) or the terms are unclear, there may be an implied (contractual) right if the employer has, through the passing of time, established a custom and practice of giving paid time off for public holidays. That is highly unlikely to apply, however, to one-off additional days such as 3 June.