Text this: Sample efficient multiagent learning in the presence of Markovian agents