But the principles are the same anytime the federal government has to decide whether to recognize a marriage. The key determinant of whether a marriage is recognized for federal taxes is whether the marriage was valid when and where it was created. If they were a mixed gender couple who resided in Ohio in 1985 and established a common law marriage, and moved later to New York, what matters for deciding their status for federal tax matters is only what happened Ohio in 1985 (absent death or a later termination by a court). New York's recognition, or lack thereof, of common law marriages doesn't even come up when you determine marital status for federal tax purposes (or social security - and likely others - those are the two I am intimately familiar with).
Where Edie Windsor acquired her marital status was not from New York, but from Canada. I would agree more with your analysis if, for example, she had been married in a state which recognize same gender marriages. Then the federal government would be taking away rights a state (other than New York) had chosen to grant. It is hard to wrap your mind around, because for mixed gender couples marriage and the recognition of marriages from another state coincide in time, so you don't normally think of the act the rights originate from - unless you have spent a fair amount of time analyzing marital status for tax or social security purposes. The analysis you work through has nothing to do with the state of residence at the time of interfacing with the tax system - it has to do with the law in the state where the marriage occurred.
(As for strong public policy - that is not just an international matter, the same concept applies to marriages in other states - and if you read the opinion carefully, you will note that the decision and later interpretations do not distinguish state marriages from international marriages. )