OOC (Out of Character) originally started out as the exploration of other sides of canon characters personalities. This was especially true for the two-dimensional characters, like Snape or Draco. We know that all people have layers upon layers of their personality - why should Snape or Draco be any different?
The problem became that people used OOC as an excuse to basically demolish a canon character. Suddenly Harry was spouting Shakespeare, Hermione was rocking the pleather mini-skirt and Draco was a really, really huge softie. It became annoying.
How to Write OOC :
Really good OOC fics explore aspects of a character. It can be so very out-there, but they work because they take a canon trait of the particular character and use it as the gateway to explore the possibilities. For example, we know that Hermione is very obsessive over her schoolwork and seems to always keep things neat. She comes across as a very tidy, prim person. It sounds like a dead end, but it's really not. Her obsession over her schoolwork can be the gateway you need. Hermione shows a need to be in control, but her friendship with Harry and Ron, and her fight to maintain her place in the Wizarding world, takes away that control. In real-life, this need to be in charge of some part of life can lead to disorders like OCD, anorexia nervosa and other serious mental disorders. Why is Hermione so obsessive over her homework? Why does she need to be the best at everything? You can write Hermione as having depression over the lack of control in her life, but we need to know why first.
Ginny is obviously very strong-willed. But why? Maybe something in her past, something that hurt her, caused a need for her to have a very dominant, loud personality. Being temperamental could be her security blanket; it might mask something. While it's unlikely that something ever happened to her, you can use her loud, fiery temper as the pathway to explore a darker side of Ginny; perhaps a need for revenge or other seemingly more sinister aspects of our bright Weasley girl.
So that's what you need in order to write a good OOC - the best OOC fictions stem from something we can string it along from. Hermione is a perfectionist, she's obsessive - where else could that lead? You need to explain where the OOCness came from. Snape is a complete and utter jerk - so why would he cry at night? What part of his personality shows weakness and vulnerability to the extent that he would weep? Perhaps his irrational anger toward Harry can reveal something. If you can present it realistically, then you've used OOC in the best way possible - it's OOC, but not unreal. We could see it happening. Make us believe. Yes, it's out of character, but even us real people have moments where we snap, and do something that we'd normally never do. The kindest girl might cuss out a fellow student; a strong hero-figure might break down and cry. The school bully has secrets about a bad family life. You just have to show us how, and why.
What Not To Do:
Never, ever, just leap into the character. Don't go "Well, Draco is really a nice guy, he just pretends to be mean!" and then launch into Sweet!Draco. That would never happen. Show us how. Draco wouldn't just wake up one day and decide that he's madly in love with Hermione. There has to be something to build upon. You need to prove that Draco has the capacity to change. Snape wouldn't suddenly come to like Harry - show us how Harry earned Snape's respect. No one does a personality turn in the space of 24 hours; there are reasons, and it takes time to change. Why would Ron turn evil? Where has it been shown that he could every truly betray his best friends? Why would he? There has to be context. Diving into a fiction and seeing Ron as the Dark Lord's right-hand man can be infuriating if the explanation is simply, "Harry and Hermione got together, and Ron went bonkers." Give us the back story. Make it interesting. Make us understand, and make us sympathise. Make us care. How long has Ron been in love with Hermione? Why did Hermione's rejection hurt so much? Who is he angrier with? Does he regret it? Does he think about it? Go over everything.
Why Severe OOCness Doesn't Work:
It doesn't work because the characters are no longer themselves. If Draco turns into a harmless bunny rabbit, he's not Draco Malfoy anymore, he's just some person who happens to look like Draco and shares the same name. When Hermione turns punk, her essence disappears. You'd be better off writing a fiction with OC's (Original Characters). Because you're working within a fandom that has a lot of character development, twisting a character's personality too far only leads to aggravation. Overused OOCs:
Attempting to write OOC is a feat that should be tried only once you have mastered the character as his/her canon-self. Never attempt to write OOC if you are not an experienced fan fiction writer. If you’re relatively new, or prefer strict canon/realism, OOC isn’t for you— and that’s okay. At the end of the day, we all need authors who know how to write Hermione and Neville and McGonagall as they are in the series; such stories are comforting. So do not be pressured.
Whether you’re writing OOC on purpose to explore a new facet of the character’s behavior, or whether you merely wish to stick to canon, it is essential to know your characters like the back of your hand. Re-read parts of the books before you sit down to write, especially parts which contain the characters you’ll be using in your own story. The tone and poise of the writing will rub off on you subconsciously. Ask yourself difficult, complex questions regarding the HP characters’ personalities. If you can’t answer them easily or adequately, do more research on that character and make sure you’ve read plenty of their canon dialogue.
For more help with characterization, we have an essay specifically on that topic; our dialogue essay also touches on the subject, so be sure to read through the rest of the Help section before sitting down to pen your personalities. Bear in mind that stories which harbor OOC characters without purpose and realism will be rejected— yet another reason to stick to canon personalities when in doubt.