You may have replaced the wrong thermostat, but more likely one, or both, element is burned out, but instead of shorting out or not working at all, it is bypassing the thermostat and overheating the water.
Now that the thermostats have been changed, we know that is not the problem. Change the element(s)if it is an electric model and you have done all that is needed. be sure to exercise caution while working on this type of equipment. ..disconect power and water to the unit before attempting any repairs. Mazel Tov
Bypassing the thermostat all by itself? Give me a break!! Make sure the thermostat is pressed against the tank. It must make contact with it to get the proper reading. Double check your wiring, then check the thermostat. Some have been known to be bad from the MFR.