My brother is deaf. He most definitely knows he sounds different. He doesn’t use ASL with me. He uses it with my sister since she is fluent. We have had our own “family slang”.