Valga, or Valka as it is known in Latvia, has more than its fair share of existential challenges. Between 1298 and 1558 it was invaded and burnt to the ground or destroyed six times. It has been colonized or occupied by Sweden and then Russia and Germany twice. Finally, from at least 1920, the town was split into two pieces and divided between Latvia and Estonia. The fences and border gates finally came down in 2009 but the town remains divided by two very different languages, one Nordic and similar to Finnish, the other one of only two still-spoken Baltic languages. As a result it is hard for people from the two sides of town to communicate and Russian is often used as the common language for those that speak it.
As with several other strange or disputed borders in this set, the people of Estonia and Latvia have a Brit to thank for the border running down the middle of their town. In 1920, Colonel S. G. Tallents helped lay out the border with most of the town going to Estonia with the exception of the area surrounding Lugazi Square.
As with other strange borders, the town's unique history has now made it something of a tourist attraction. The Visit Estonia website asks, "Where else could you stand, one foot in one country, holding “jäätis” (ice cream in Estonian) in your left hand and other foot in another country, holding “saldejums” (ice cream in Latvian) in your right hand?”
Thanks to Käty Tarkpea for alerting me to this border curiosity
Sources: Geosite , Wikipedia , Visit Estonia