I would recommend that you first find another seat. Particularly around a pool, people often spend time in the pool and, if it's busy, you may not know who the items belong to and they may be nearby, just swimming. There's no reason to move someone's items if there are other seats available.
If there are no seats available, take stock of what the items are (without being too nosy) - are they personally owned items (a book, sunglasses, purse) or public items (beverage glasses or hotel towels)? If the items are public items, they may have been left behind intentionally and the people are not planning to return. Next, see if the people sitting nearby know who owns the items and how long they've been away. If it's been a long while, you're probably safe to use the seats. If it's only been a few minutes, I'd try to find another spot.
If there are personal items and you decide to take over the space, be respectful of them. Group them together on a nearby table or turn them in to staff for the lost and found. Keep the name of the person you turned them over to so that it's easier for the owners to reclaim their items.
You might also see if the venue has any policy on this. Some hotels may have rules about reserving seats without occupying them or they may be able to help you find out if the owner still plans to return.
If you start to take the spot and someone comes from the pool to let you know that it's their stuff, you might ask if they mind if you sit there while they're in the pool or you can move on to finding a new seat.
If you take the spot without someone stopping you and the owner returns a while later, I don't think you need to feel obliged to give up the spot. I would hope that they would realize that, without leaving a representative there to "save" the seats, they'd likely be taken by someone else, particularly during a busy time. Unfortunately, it's not uncommon that the very people who think this practice is OK are also the people who think they can throw a fit when you take "their" spot. It's a public seat and they weren't actively using it. It's perfectly acceptable for you to make use of it when they are not.
Be on the lookout for alternate seating and either point them in that direction or be willing to move yourself if they get too vocal. Please, if they're unreasonable, don't be afraid to talk to the staff. That's part of why they're there. They may be able to offer a solution.