How can I improve my singing voice without formal singing lessons?

I have the same problem as you. All you need to do is practice, practice, practice. When you sing different songs, you'll be able to see how far your voice can go without straining and help you get better. You also need to practice breathing techniques, which you can always Google. You don't always need to hit the big notes if it makes your throat hurt. Just stick with your comfortable vocal range and your voice will improve.

Okay, first of all, find out your voice type. If you're singing out of your range you could do some serious damage to your vocal cords. There are three female voice types: soprano, mezzo-soprano, and contralto. A soprano is the highest female voice, sopranos are comfortable singing high notes, they have a light sweet timbre to their voice, but they can also be very powerful, some examples of sopranos in popular music are Hayley Williams (of Paramore), Christina Aguilera, Celine Dion. Then there's mezzo-soprano, it's some where in between a soprano and contralto, it is the most common female voice, Madonna, Lady GaGa, Gwen Stefani, Beyonce, they're all mezzo-sopranos. And then contralto is the rarest female voice, it's very deep, often mistaken for a male's voice, contraltos feel a lot more comfortable singing from their chest, although sopranos prefer singing from their head, Nina Simone, Mariah Carey, Alicia Keys, Cher, Toni Braxton, Fiona Apple, Amy Winehouse, they're all contraltos. Now, from that information could you guess your voice type? It'll be a lot easier once you know your proper classification, that way you won't damage your voice from singing out of your range. It's actually quite easy, just do it like this - if you sound like Christina Aguilera, you're most likely a soprano. If you sound like Madonna, Lady GaGa, if you basically have an average sounding voice you're a mezzo-soprano. And it should be really easy to know if you're a contralto, just ask yourself “do I sound like a man when singing?”, lol, it's that easy. Breathing exercises are important, you can easily find them free online. Also look for warm-up routines, things like that. Good luck.