No, it's the higher skill. So hill 20 will go down to 15 in this case.
Edit: the wiki also seems to argue it's a random one though, so now I'm not fully sure anymore. I usually bring one skill to 20 right before the decrease (and keep others at 19) and I think the 20 rated skill always went down, but I might be mistaken?
In Joelag 's case he will decrease in Hill, if both Hill and Mountain would be 18 f.i. then a (random) choice will be made between the two skills (or three skills or ... whatever number of skills are equally highest)