Desmond and Olivia are both tired of Layla’s antics