To deal nonlethal damage in DnD 5e, you simply have to reduce a creature to 0 hit points with a melee attack. The moment you deal the damage, you decide that the attack will knock the creature out instead of killing them.
At that moment, the target falls unconscious and is stable (no death saves).
An unconscious, stable creature that isn’t healed remains unconscious for 1d4 hours. At that point, it wakes up with 1 hit point.
Can You Deal Nonlethal Damage With Ranged Attacks?
No, you cannot deal nonlethal damage with ranged attacks in 5e. If you bring a creature to 0 hit points with a ranged attack, regardless of whether the attack was physical or magical, the target dies (or starts making deaht saves, at the DM’s discretion).
Can You Deal Nonlethal Damage With Spells?
Yes, you can deal nonlethal damage with melee spell attacks. However, there are very few melee spell attacks in the game:
-
Flame Blade
-
Vampiric Touch
-
Contagion
-
Mordenkainen’s Sword
-
Blade of Disaster
Note that Thorn Whip has a 30-foot range, but is still a melee spell attack; an oddity of the spell that makes it the only way for a player to deal nonlethal, knockout damage at range.
This has been confirmed by the game’s devs in the Sage Advice Compendium:
Player’s Handbook, page 198