When I click on "Show hidden icons", then the hidden Steam icon, I see a list of three items at the top of a drop down menu, from upper to down: Doctor Kvorak's Obliteration Game Demo, Team Fortress 2 and Pink Hour. I'd like to get as much of them removed from that list as possible. How can I do that? I'm using Windows 10, just in case that'll help you answer my question.
I guess you mean the 'Show hidden icons' button ('^') on your Windows taskbar, and the Steam "quick menu":
It shows the games you have recently played or acquired (I've not played RobocraftX, for example), so, if you want to get rid of those specific items, try playing or downloading other games.