Abstract: We investigate the dual approach to dynamic programming and reinforcement learning, based on maintaining an explicit representation of stationary ... we present a modified dual of the ...