The scent on your hands will rub off on the baby hamster. The mom will think that the hamster is not her baby and she will abandon them or sometimes even eat them.