The first evidence I have found so far for James, is in the 1840 Philadelphia census. For the most part, the family remained rooted in Philadelphia but seemed to move back and forth between Wilmington, Delaware and Philadelphia, Pennsylvania. While in the Philadelphia area, this family tended to migrate from just south of the old city up through the Fishtown and Kensington areas to Port Richmond. More recently the remaining family seems to have moved to Northeast Philadelphia and Bucks County.
Front Street between Race & Vine - 1868 Reprinted in: Old Philadelphia in early photographs, 1839-1914/edited by Robert F. Looney. New York: Dover Publications, c1976. Fitzgerald's grocery shop, with its firemark between the upper windows, stands in a group of Colonial buildings among the oldest in the river-front area. These buildings are just inside the northern boundary of the old city. --OPEP, p. 38.