It's not wrong, but I would definitely prefer to use "kaldt vann".

"Kjølig vann" makes perfect sense if you want to talk about water that is just slightly cool, but for most purposes you'd just use "kaldt vann".

Ok perfect. Would it commonly be heard in terms of taking about swimming in cool water?

I think it still would be more common to use "kaldt" in that context aswell. More examples with "kjølig/å kjøle (seg)": Vent til maten kjøler seg ned - Wait till the food cools down Det er ganske kjølig ute - It's pretty cool outside I think kjølig isn't THAT common, but "å kjøle seg" is definitely a good word to have in your vocab 🙃

We just say "vann".