We inherit a sinful nature, not sin itself. Ever since sin entered into the world, it was man's nature to sin. However babies can't judge a right action from a wrong action, so the nature of man to sin doesn't affect them. So babies can't go to hell because they can't sin