Email Record: Sample efficient multiagent learning in the presence of Markovian agents