Whenever I see a strange border like this, the bonus section always seems to belong to the more powerful of the two neighbors. Marble Hill is technically part of Manhattan even though it is now connected to the Bronx and separated from Manhattan by the Harlem River. If the Bronx was the more dominant player, I wonder whether the border would now follow a more egalitarian course.
When the Dutch first settled Manhattan, they called the area where the Harlem River emptied into the Hudson River, “Spuyten Duyvil”, the spouting devil due to its treachorous currents. The area contained shallow shoals and a tricky section of the Harlem River that curved around Marble Hill. It was hard to navigate in small boats let alone the steamships that plied these waters in the latter half of the 1800’s. So, in 1890, work was started on the Harlem River Ship Canal which cut off the need to go around Marble Hill. The former course of the river was then filled in to connect it to the rest of the Bronx. As a result, Marble Hill went from being part of the island of Manhattan, to being its own island, to finally being part of the Bronx. The border around Marble Hill follows the course of the former river.
Sources: Wikipedia