A Volatge Regulator is a component designed to provide a constant specific output voltage, this can be higher, lower or the same as the input voltage. They're often used when a source voltage cannot be connected directly to a circuit or subcircuits need to be powered from the same power source but run on different voltages. They also allow you to use/power circuits using power sources with unstable or constant voltages. The most common use is to drop or raise the voltage to either 3.3V or 5V. A great example of a voltage being regulated would be the USB plugs commonly used with most current smart phone chargers, this regulates the 230V from the mains down to the 5V needed to charge the phone.
5V Step-Up/Step-Down Voltage Regulator
Step-Up vs Step-Down
There are linear regulators that output the same voltage as the inputted voltage, these are often used when there is likely to be changes in load that would usually effect a voltage supply. There are also switching regulators that are capable of outputting a higher voltage than inputted as well as the more common version which lowers the voltage. If the regulator lowers voltage it is referred to as a Step-Down or buck regulator, if it increases the voltage it's referred to as a Step-Up or boost regulator. There are also some more complex regulators that can perform both functions, these are called Step-Up/Step-Down or Buck-Boost regulators.
Adjustable 4-12V Step-Up/Step-Down Voltage Regulator