Will running a test longer make the significance go above 95%?

No, running a test for a longer period of time won't automatically make the results significant. It does however increase the statistical likelihood that the results are correct.