Don't both change directory and set GIT_WORK_TREE