Spending some perk points in speechcraft doesn't really take away from your character's overall combat ability if you plan your spending way ahead of time. Even on Master level, my Redguard is virtually unstoppable.
I went the two-handed and heavy armor route. I went all the way to the top perks in both. I use a warhammer as my primary weapon. Thing deals 630+ damage with two enchantments. Armor rating is over 1300. My restoration ability is very respectable.
I pretty much focused on getting the right perks in two-handed, heavy armor, enchanting and alchemy in order to walk around and worry about nothing. I can punch a giant to death in a matter of seconds if I want lol.
I use alchemy to create ridiculous potions to help me craft and enchant better weapons and armor. Then the two-handed and heavy armor perks take over from there.