The Secret Service is investigating the matter, the White House said. "They're checking visitor logs and ... looking at cameras. Those are the next steps. Cross-checking," said the source.
White House spokesperson Karine Jean-Pierre told reporters: "Where this was discovered is a heavily traveled area where many ... West Wing visitors come through."
Asked whether anyone had undergone drug testing as part of the investigation, Jean-Pierre said: "We will take any action ... that is appropriate and warranted, pending the outcome of Secret Service."
Possession of cocaine, which is classified as a "schedule II" drug under the U.S. Controlled Substance Act, is a misdemeanor in Washington, D.C.
The West Wing is attached to the executive mansion where President Joe Biden and first lady Jill Biden live.
It includes the Oval Office, cabinet room and work space for presidential staff. Hundreds of people pass through the West Wing on a regular basis, including political staff, their guests and members of the press.
Jean-Pierre said West Wing tours took place on Friday, Saturday and Sunday.
The substance was found during a routine Secret Service sweep on Sunday evening. A Secret Service spokesperson did not respond to requests for comment on Wednesday. The agency has not said how much cocaine was found.
The discovery led to a brief closure of the White House complex on Sunday. Biden and his family were not at the White House then.
Biden did not answer questions that reporters shouted at him about the cocaine on Wednesday. The president thinks it's "incredibly important" to get to the bottom of it, Jean-Pierre said.
Administration officials are able to offer tours of parts of the West Wing to friends and family members. People who are not members of staff must leave electronics and other belongings in the storage cubicles before taking a tour.
"It was in one of the cubbies," the source said about the cocaine.
Biden and his family returned to the White House early on Tuesday after spending the holiday weekend at the presidential retreat at Camp David, Maryland. Family members who visit the Bidens traditionally enter and exit through the East Wing, the source said.