Can points assigned to a user be recovered?

Yes. The process is identical to the assignment but selecting the "Remove points" option (either manually or by file).

The points are returned to the wallet so they can be reassigned.

If the assignment was scheduled and the date has not yet passed you can remove the user from the schedule from the "View schedules" list.