tools: buildman: Don't use the working dir as build dir