Direct debit bounced twice and now I'm being charged a fee?

Hi all, hoping someone can make sense of this because I’m a bit fed up. My Nexora direct debit failed last month even though there was money in the account, then it’s just failed again today. I’ve checked with my bank and they say the instruction looks active. Nexora have now added a £5 missed payment fee and I’ve got a text saying service may be restricted if I don’t pay by card. I’ve not changed banks, card, address, nothing. Does anyone know if this is a known issue or am I missing something obvious? I’m on Fibre 150 and my bill is usually about £31.50. Account ends 4821, if that helps. Cheers.