By the time you are done with the swamp, greydwarves are so trivial it stops mattering and its just free resources. I was a bit sad when they patched out the greydwarf farming though.