Being an atheist here is no problem at all. You can be an atheist in public no trouble whatsoever.You keep it that way! Is it unusual on your island to be an atheist? I don't know how religious the people of Sri Lanka are.
However there are a few things worth mentioning. Buddhists take offense when they see something like a Buddhist flag worn as a garment, or stuff like that. They never assault or anything but they don't like it and may scold you sometimes. A few years ago a tourist who had a Buddha tattooed on arm was deported by the authorities after a complaint made by some taxi drivers. Then this went public and the general public protested against that treatment. Then the government gave her compensation and invited her back totally free and offered her an absolutely free tour. I don't know if she came back. (I wouldn't)
In 2019 suicide bombers carried out attacks on several churches on Easter Sunday killing 300 people. They said it was against the Christchurch shooting in Australia.
p.s. The Esterday bombing was carried out by the Islamists pledging allegiance to ISIS.