It was not clear what Trump hoped to convey with his "team" comment or how Rosenstein might have interpreted it.
"Of course, we're all on your team, Mr. President", Rosenstein told Trump, the sources said.
White House adviser Kellyanne Conway confirmed in a May interview after Comey was sacked that loyalty was an important part of the White House culture.
Trump reportedly changed the topic of conversation to the status of Special Counsel Robert S. Mueller III's investigation.
During his testimony before the House Judiciary Committee, Rosenstein denied that he had ever been asked to be loyal to the administration. "I think that there are a lot of people in American - especially in my district of Kentucky - that are disappointed int he attorney general and Rosenstein, and you know, if there is something there with Russian Federation, then let's get it out... this whole Russian Federation investigation has been a big distraction".
Rosenstein reportedly is at the center of the memo, authored by House Intelligence Committee chairman Rep. Devin Nunes, over his approval to extend a Foreign Intelligence Surveillance Act (FISA) warrant on Carter Page, former campaign aide to then-candidate Donald Trump.
So it's significant that on Tuesday, Rep. James Comer (R-KY) went on CNN and said he would have no qualms if Trump chose to remove Rosenstein. It does not appear those questions were asked of Rosenstein at the hearing.
But Republicans have urged Trump not to hastily dismiss Rosenstein due to the political optics and due process, according to The Hill.
"Nobody has asked me to take a loyalty pledge, other than the oath of office", Rosenstein testified.
Senator Jeff Flake (R-Ariz.), who has routinely clashed with and criticized Trump, suggested that firing Rosenstein would provide the president the same dire results as former FBI Director James Comey's abrupt firing.
Rosenstein, who appointed Mueller in May after Trump fired Comey, has faced the president's wrath throughout his first year in office. He also pushed back at Democrats' questions about the President's demands for loyalty.