There has been a stickiness to the electorate's concerns about issues of immigration and asylum, Chris Mason writes.