Brunch on the Upper East Side is not a scene, and that's the point. There's no line down the sidewalk, no DJ, no bottomless anything at most of these places. What you get instead is a table you can sit at for two hours, coffee that keeps coming, and a neighborhood that's mostly out walking its dogs.
We've sorted this list by walk time from our door on East 62nd Street, because that's the only measurement that matters on a Sunday morning. Anything in the first group you can do in slippers. Anything past 79th Street is a real walk, and worth it if you're heading to the museums after.
One piece of advice we give everyone: the East Side eats early. If you show up at 9:30 you'll walk into almost any of these. If you show up at 12:15 on a Sunday without a reservation you'll be standing on Madison Avenue reading a menu you can't get a table for. Book the popular ones (Sant Ambroeus, Cafe d'Alsace, Amaranth on a weekend) two or three days out and you're fine.
You also have a kitchen in your apartment here, which changes the math. Plenty of our guests do coffee and eggs at home on day two and save the sit-down brunch for the day they've got somewhere to be afterward. Both are correct.