to taki skrot od calego wyrazenia: my brother's (house/home/place), lub jakikolwiek inny rzeczownik opisujacy miejsce zamieszkania/pobytu osoby, do ktorej sie udajemy
I've found that it is easier to understand if you assume that when the ('s) is there it is something belonging to that person.
My brother's dog, my sister's doll, my mother's apron, my father's car.