fix mishandling of ENOMEM return case in internal getgrent_a function

fix mishandling of ENOMEM return case in internal getgrent_a function

due to an incorrect return statement in this error case, the
previously blocked cancellation state was not restored and no result
was stored. this could lead to invalid (read) accesses in the caller
resulting in crashes or nonsensical result data in the event of memory
exhaustion.