In the middle ages people would travel vast distances to worship and pray in the presence of holy relics at sites such as St Andrews in Fife. Ever since St. Rule was washed ashore with the remains of St Andrew in 365 AD, the town has been a place of Christian teaching and worship. The Firth of Forth proved a barrier for pilgrims travelling from the south and a ferry was established in 950 AD between North Berwick and Earlsferry in Fife. The town of Earlsferry was named after MacDuff, Earl or Thane of Fife who also owned lands at North Berwick and used a ferry to cross the Firth of Forth. The Kings and Queens invested heavily in promoting the monastery at St Andrews. They also built the church of St. Rule, its enormous square tower was like a beacon and could be seen for miles around.

To serve the needs of the pilgrims using the ferry at North Berwick a hospice and church were built. The ruins of the church can be seen on the Anchor Green and the hospice was situated to the north west of the church. The nuns from North Berwick Abbey also looked after the hospice at Earlsferry. Later a hospital for the poor which the Lauder family were patrons was sited in the area presently occupied by the Housing Association flats in Quality Street. The journey to a shrine was not only spiritual but a holiday, particularly for the peasant farm workers as their landlord was obliged to grant time off work to take part in a pilgrimage and the church looked after their procession while they were away.

Pilgrims meant money, they were the tourists of their day, producing prosperity in their wake in souvenirs and trade at the market stalls. A clay mould for casting lead pilgrim badges was found at the Auld Kirk and is now in the North Berwick Museum. The badge decorated with the cross of St Andrew had loops attached whereby the badge could be sewn onto the pilgrims clothing as a token they had undertaken the arduous ferry crossing, with high seas and the Fife coast often shrouded in thick mist. The movement of around ten thousand pilgrims a year passing through this tiny little village was an outstanding feature for many years. The ferry depicted in the North Berwick Coat of Arms was also regularly used by Scottish Kings such as James VI in 1592.