I'd like to add on to what Terri said about Fudge being in Slytherin. I agree with you whole heartedly, and I'd just like to expand on your points about why he couldn't have been in any of the other three houses.
Hufflepuff is a no-go to begin with just because of what happened to Sirius - he was sent to Azkaban without a trail. This may or may not have been at Fudge's orders (I'm under the impression that Barty Crouch Senior took control of most trials) but he defiantely didn't do anything to fix it once he was informed, even though he must have known that it was immensly unfair. Another point (well, really, two) would be in the very first chapter of HBP when he is informing the British Prime Minister about the return of Voldemort.
"'My fault!' said Fudge, coloring up. 'Are you saying you would have caved in to blackmail like that?'"
Originally Posted by Harry Potter and the Half Blood Prince, US Edition, Page 12
"'My dear Prime Minister, you can't honestly think I'm still Minister of Magic after all this? I was sacked three days ago! The whole Wizarding community has been screaming of my resignation for a fortnight. I've never known them so united in my whole term of office!' said Fudge, with a brave attempt at a smile.'
Originally Posted by Harry Potter and the Half Blood Prince, US Edition, Page 15
In my opinon, both of these quotes show that Fudge isn't willing to see himself as someone who caused the deaths of many and tries to defend himself to all end (which, in my view isn't a Hufflepuff trait - I've always thought of them as willing to be blamed for what they did but never standing for being accused of what they didn't do). This, to me, is more of a selfish, ambitious trait (or just self-preservation) that seems to fit best in Slytherin rather than in any of the other houses.
Ravenclaw is out for one reason only: if he was smart, or at least recognized himself with having an ounce of intelligence, then he would have been able to see that it wasn't right to let Sirius go without a trial, or it wasn't safe to let the larger part of the Wizarding World go on believing that there was no return of Voldemort... and the big one: not heeding Dumbledore's advice. If he didn't listen to the Wizard who was supposed to be one of the most intellegent in the entire Wizarding World, then who would he listen to?
Gryffindor isn't possible because of what Terri has already mentioned - he either didn't feel like he could stand up to Voldemort, or he wasn't brave enough as a person or a leader to stand at the forefront of the Second War.
Now that I've typed up a lot about his traits, I'm starting to wonder about his wife, since he was married. What do you think their marriage was like (before, during, and after the Second War)?