I like to put them on a bed and let them run around and when they are ready to stop. I pick them up and play with them. They sort of have to investigate every area before they are ready to let their guard down. Be patient... Hamsters are just wired like that!
If your hamster is young, they tend to be hyper. As they get older, they will start to calm down a little. But if it's still happening, put him/her in a ball, get them a wheel in their cage, just try to find out ways to make her tired.
Put it into a free roam ball for 3 times a day, 30mins a time, and talk to it at day.. This will wear it out.