I agree with Horatio- Also if you do try again with a different male keep in mind the pups and cage should not be disturbed and only food and water should be changed until the mother has weaned them and they are eating food on their own, also with some species the male should be removed because they can harm the babies. A female hamster should not be pregnant constantly either it can cause her to die and have complications in birth as well as her having poor offspring.
Incest causes genetically inferior off-spring. If the mother keeps having babies, she will keep eating them. You should never have opposite sex siblings together in the same cage. If the mother gives birth too many times, she will die. Please separate the siblings.
Usually hamster incest will go fine for the first litter, the second litter, the babies might be still born or be deformed.