We cannot answer baseless allegations - Galle District Secretary