Meta: Recurring Villains

Another meta post for today. This time on recurring villains! I like the idea of recurring villains, but all too often they seem very contrived in games. This is just a write-up of how I think they should and should not be done, and to what ends they should be used. 

Recurring villains are something that's pretty difficult to have in D&D games. The party expects to kill any boss, and the way the dice fall can completely mess up any plans that a DM lays. You should never script that an NPC escapes from the party, at least not without giving them the chance to intercept and cut them down. So it's very disheartening to have a big bad guy and expect them to return, only to have them die in the first round to a lucky crit. You spend all this time developing rich backstories and interesting campaign arcs with adventure hooks, and then it's all cut short before any of it has the chance to get off the ground. 
The important thing to remember is that sometimes it's okay to make the arbitrary decision to just not let an NPC die. You can have them resurrected, you can have them use a one-time magical ability that you just invented, on the fly you can adjust their hit points to survive long enough to enact their evil plan. The trick is to not overdo it and make your players feel like they aren't accomplishing anything because the bad guys are always getting away.
So first what I want to talk about is: what kinds of villains should be recurring, and which ones should die without any big entrance? In my opinion, recurring villains should be like Bowser. You fight them multiple times, and each time they add new mechanics. They have the same basic theme, like fire or undead or whatever. But they gain new spells and abilities each time the players fight them. That adds a new level of knowing roughly what they're getting into, but the players always know that something may surprise them.

Villains with a good reason to recur:

• They contribute to the plot of the characters
• They can be woven into the backstories, or have been encountered during play
• They should have a personal reason to antagonize the party
• They have the means and determination to have plans to ruin the party
• A campaign-driving villain
• The elder being from the depths, the agent of chaos, the lord over all undead. For lack of a better term, the Big Bad Evil Guy.
• They should start out stronger than the players. This gives a way to realistically have them not be defeated when first encountered.
• They have big plans. Most times they won't even be encountered. Their lackeys can die easily while the plan still follows through.

Villains with a bad reason to recur:

• Your personal favorite
• I know that I have some really cool concepts that I would love the party to deal with forever and ever. But that gets really boring, really fast. If they don't contribute anything other than scratching an itch for you, it's not worth it.
• NPC favoritism is toxic. If you make the whole story about your favorite NPC (which I know I at least am guilty of) then the whole thing suffers. If the players aren't interested, move on.
• A minor villain
• Most minor villains are minor for a reason. Let them have their fight and die, and then move on.
• Sometimes it might make sense. They may swear a vendetta against the party, but let it wait for a long long time before they resurface.

Good ways for villains to recur:

• The PCs are actively fighting against them
• This means that the villain is orchestrating some evil plan which the PCs are thwarting at every turn.
• This gives the villain a plot-fulfilling reason to exist, and enough wiggle-room for them to recur.
• They are intelligent and plan for every outcome
• This one's a little more tricky. You don't want to fall into the trap of making up reasons for them to survive, but if a villain is reasonably intelligent and powerful, they should have some fallback plans. 
• If the villain is aware of the party's intentions, they can obviously prepare for an inevitable showdown, and plot their hasty retreat.
• A wizard did it, with context
• If an evil sorcerer is slain and comes back later as an evil lich, that makes sense. If an evil dragon becomes a Dracolich, that makes sense. If a fallen Paladin returns as a Blackguard, that makes sense.
• Magical reasons for reanimation and a fall towards evil are perfectly okay as long as it fits the plot of that villain, and is rare.

Bad ways for villains to recur:

• "You never searched for the body!"
• This is awful. If the PCs kill someone that you liked, don't just bring them back with no good reason.
• This sets the precedent that nothing the PCs do is final, and out of their control. Bad.
• A wizard did it, without context.
• There's a reason that "A wizard did it" has such a negative air about it.
• Again, inventing random magical reasons is almost never a good idea.