Look at their faces when he says this. They look so sad because they realize that they never showed Sherlock how much they cared for him. Yes, Lestrade is manly but why should that stop him from showing Sherlock how much he cared? Molly tried to show Sherlock how she felt but he never really got it. These two consider Sherlock to be their friend, and it hurts them to find out that Sherlock didn't know it.