He'll always be remembered (at least by me) for his porn-star mustache and sweeping slider. Jeff Nelson, a huge part of the Yanks' 4 rings in 5 years, signed a minor league contract with the Yanks, then retired cementing his place in Yankee history.
Say what you want about Nelson falling in love with his slider, walking too many guys, being surly, but you just can't ignore the fact that the Yanks have been looking for a bridge from the starter to Rivera ever since he left after the 2000 season (not coincidentally the last WS the Yanks won). Nelson and Stanton combined to get the Yanks through the 7th and 8th innings and to Mo with the lead. He was one of the few guys Joe Torre has ever trusted coming out of the pen, and even with all the headaches, he'd take a guy like Nelson was in his prime every year.
Nelson bounced around after leaving the Yanks, only putting up one more solid season in 2001 with Seattle. His career will be largely forgotten by everyone who wasn't a Yankee fan during those magical years, but for us he joins guys like Scott Brosius and Paul O'Neill as winners we'll never forget.