Mr Rosenstein verbally resigned to White House Chief of Staff John Kelly, Axios reported, citing an unidentified source with knowledge of the matter.
A second source told Axios that Rosenstein is “expecting to be fired” so he plans to step down.
Donald Trump faces mounting pressure from the investigation by special counsel Robert Mueller, who is looking into Russia’s role in the 2016 presidential election.
Mr Rosenstein assumed supervision of the probe after his boss, Attorney General Jeff Sessions, recused himself because of his own contacts with Russia’s ambassador to Washington while serving as a Trump campaign adviser became public.
There was widespread speculation that Trump would fire Mr Rosenstein after a New York Times report on Friday said in 2017 he had suggested secretly recording Trump and recruiting Cabinet members to invoke a constitutional amendment to remove him from the office.
The Times said none of those proposals came to fruition. Mr Rosenstein denied the report as “inaccurate and factually incorrect.”
The move comes just six weeks ahead of the November 6 congressional elections, and could become an explosive political issue as Mr Trump’s fellow Republicans try to keep control of Congress.
Mr Trump has regularly dismissed the investigation as a “witch hunt.”