A source tells PEOPLE that Cheryl Hines and Robert F. Kennedy Jr. have "barely spoken" in the two weeks since his alleged "personal relationship" with reporter Olivia Nuzzi was revealed by New York magazine.

Amid reports that Cheryl Hines and Robert F. Kennedy Jr. “have barely spoken” in the weeks since news broke of an alleged affair involving him and a much younger political reporter, sources are now saying the rumored relationship is not the central focus of the couple’s problems.

As Robert F. Kennedy Jr.’s turbulent year continues, his wife, actress Cheryl Hines, has stayed quiet. Kennedy, 70, dropped out of the presidential race in late August, choosing to back former President Donald Trump.